Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 674.701942ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 674.731456ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 407.102137ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:50:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 407.124834ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 423.853277ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:49:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.877835ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 486.610006ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:49:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 486.633937ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 377.161546ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:48:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 377.189972ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 435.115204ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:47:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 435.143897ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 423.396521ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:46:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.475563ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 782.057783ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:45:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 782.094187ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 480.56654ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:45:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 480.624027ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 423.129583ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:44:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 423.154874ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 667.107731ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:43:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 667.128773ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.039941428s trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 18:43:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.039972155s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 273.845592ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:38:05 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 273.870163ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 276.47582ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:37:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 276.511671ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 285.715276ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:36:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 285.751046ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 322.639839ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:36:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 322.677257ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 267.255298ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:35:40 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 267.280934ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 279.812755ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:34:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 279.894136ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 275.250804ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:33:59 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 275.28035ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 481.731536ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:33:22 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 481.813938ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 260.791671ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:32:27 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 260.813967ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 360.843721ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:31:42 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 360.874759ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 330.931033ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:30:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 330.953876ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 327.910871ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:30:24 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 327.952548ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 943.482874ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:30:09 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 943.514121ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 276.268174ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:29:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 276.294185ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 272.49748ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:29:27 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 272.523157ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 309.795951ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:28:43 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 309.833502ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 770.913455ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:27:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 770.939339ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 334.754033ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:27:19 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 334.7853ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 274.978593ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:26:20 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 274.998769ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 290.834971ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:25:52 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 290.854565ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 275.301932ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:25:22 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 275.316379ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [X....................X......................................] duration: 793.851472ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:24:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 793.885822ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.X....................X.....................................] duration: 277.662562ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:24:14 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 277.688727ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [..X....................X....................................] duration: 265.078553ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:23:03 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 265.105424ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [...X....................X...................................] duration: 270.2766ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:22:48 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 270.304885ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [....X....................X..................................] duration: 443.551585ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:21:38 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 443.654342ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.....X....................X.................................] duration: 343.682032ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:20:25 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 343.713243ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [......X....................X................................] duration: 962.61469ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:19:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 962.641056ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.......X....................X...............................] duration: 277.867083ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:18:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 277.894937ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [........X....................X..............................] duration: 497.186471ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:17:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 497.222095ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.........X....................X.............................] duration: 268.248379ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:16:50 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 268.277268ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [..........X....................X............................] duration: 268.244923ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:15:50 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 268.346916ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [...........X....................X...........................] duration: 782.965351ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:14:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 782.990835ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [............X....................X..........................] duration: 274.468595ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:13:48 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 274.493614ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.............X....................X.........................] duration: 331.753425ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:12:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 331.779623ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [..............X....................X........................] duration: 370.105497ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:12:38 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 370.132334ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [...............X....................X.......................] duration: 359.925638ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:12:13 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 359.957432ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [................X....................X......................] duration: 409.533074ms remote: "tcp:136.0.77.4:443" target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/432e4bafa7e53c9b FAIL (2026-01-26 18:11:12 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 409.68957ms edge linksharing bgp ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)