Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 616.535977ms FAIL (2025-07-15 07:24:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 616.56549ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 520.360024ms FAIL (2025-07-15 07:24:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 520.39172ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 554.406799ms FAIL (2025-07-15 07:24:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 554.433925ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 498.239989ms FAIL (2025-07-15 07:23:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 498.270928ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 677.887959ms FAIL (2025-07-15 07:22:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 677.917294ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 610.16278ms FAIL (2025-07-15 07:22:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 610.194741ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 466.884754ms FAIL (2025-07-15 07:22:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 466.918616ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 429.354773ms FAIL (2025-07-15 07:21:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 429.393733ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 530.809572ms FAIL (2025-07-15 07:21:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 530.874045ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 481.482778ms FAIL (2025-07-15 07:20:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 481.514875ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XXX........................................................] duration: 452.227709ms FAIL (2025-07-15 07:20:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 452.25968ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 527.529659ms FAIL (2025-07-15 07:19:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 527.562643ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XXX.......................................................] duration: 446.861588ms FAIL (2025-07-15 07:19:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 446.893201ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XXX......................................................] duration: 500.918063ms FAIL (2025-07-15 07:19:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 500.936405ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 519.723013ms FAIL (2025-07-15 07:18:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 519.751813ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XXX.....................................................] duration: 466.058685ms FAIL (2025-07-15 07:18:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 466.091383ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 618.026843ms FAIL (2025-07-15 07:17:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 618.059062ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 612.082001ms FAIL (2025-07-15 07:17:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 612.10739ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration: 674.006528ms FAIL (2025-07-15 07:17:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 674.030994ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 550.642396ms FAIL (2025-07-15 07:17:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 550.675924ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XXX...................................................] duration: 486.338916ms FAIL (2025-07-15 07:16:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 486.3666ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 487.665271ms FAIL (2025-07-15 07:16:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 487.717757ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XXX..................................................] duration: 845.546597ms FAIL (2025-07-15 07:16:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 845.574969ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 631.331391ms FAIL (2025-07-15 07:15:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 631.36861ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XXX.................................................] duration: 496.470561ms FAIL (2025-07-15 07:15:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 496.503499ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 498.958151ms FAIL (2025-07-15 07:14:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 498.994087ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XXX................................................] duration: 440.767956ms FAIL (2025-07-15 07:14:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 440.798553ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 430.398447ms FAIL (2025-07-15 07:14:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 430.429619ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 501.456874ms FAIL (2025-07-15 07:13:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 501.493614ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XXX...............................................] duration: 390.386332ms FAIL (2025-07-15 07:13:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 390.41964ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 591.726773ms FAIL (2025-07-15 07:13:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 591.756949ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XXX..............................................] duration: 519.344022ms FAIL (2025-07-15 07:12:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 519.382656ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............XXX.............................................] duration: 670.000038ms FAIL (2025-07-15 07:12:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 670.103663ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 561.441827ms FAIL (2025-07-15 07:12:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 561.472859ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 502.938139ms FAIL (2025-07-15 07:11:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 502.965871ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............XXX............................................] duration: 477.668243ms FAIL (2025-07-15 07:11:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 477.710923ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 542.086189ms FAIL (2025-07-15 07:11:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 542.114416ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 379.297352ms FAIL (2025-07-15 07:10:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 379.328943ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration: 451.161285ms FAIL (2025-07-15 07:10:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 451.193549ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 557.805294ms FAIL (2025-07-15 07:10:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 557.896055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............XXX..........................................] duration: 455.534706ms FAIL (2025-07-15 07:09:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 455.563683ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 540.74273ms FAIL (2025-07-15 07:09:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 540.77876ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................XXX.........................................] duration: 469.450812ms FAIL (2025-07-15 07:09:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 469.479124ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 648.650249ms FAIL (2025-07-15 07:08:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 648.70808ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................XXX........................................] duration: 422.937949ms FAIL (2025-07-15 07:08:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 422.968986ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 701.956691ms FAIL (2025-07-15 07:07:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 702.001901ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................XXX.......................................] duration: 507.013797ms FAIL (2025-07-15 07:07:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 507.040275ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................XXX......................................] duration: 464.135829ms FAIL (2025-07-15 07:06:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 464.240202ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 602.299134ms FAIL (2025-07-15 07:06:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 602.400115ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................XXX.....................................] duration: 541.715944ms FAIL (2025-07-15 07:05:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 541.754956ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)