Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 508.861464ms FAIL (2025-07-15 06:50:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 508.895098ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 495.261335ms FAIL (2025-07-15 06:49:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 495.284504ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 515.129642ms FAIL (2025-07-15 06:48:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 515.161914ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 415.081905ms FAIL (2025-07-15 06:48:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 415.108292ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 707.617184ms FAIL (2025-07-15 06:48:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 707.648711ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 632.932337ms FAIL (2025-07-15 06:48:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 632.962623ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 487.34745ms FAIL (2025-07-15 06:47:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 487.375865ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 503.021876ms FAIL (2025-07-15 06:47:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 503.05267ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 532.337372ms FAIL (2025-07-15 06:46:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 532.360264ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 733.547262ms FAIL (2025-07-15 06:46:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 733.584473ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 593.550429ms FAIL (2025-07-15 06:45:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 593.585895ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 543.245899ms FAIL (2025-07-15 06:44:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 543.271696ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 568.976825ms FAIL (2025-07-15 06:44:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 569.036322ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 428.099388ms FAIL (2025-07-15 06:44:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 428.132474ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 495.801211ms FAIL (2025-07-15 06:43:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 495.840909ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 435.472054ms FAIL (2025-07-15 06:43:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 435.499035ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 781.673005ms FAIL (2025-07-15 06:43:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 781.701705ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 446.80223ms FAIL (2025-07-15 06:42:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 446.833197ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 502.798793ms FAIL (2025-07-15 06:42:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 502.874888ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration: 406.83035ms FAIL (2025-07-15 06:42:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 406.858076ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 634.412493ms FAIL (2025-07-15 06:42:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 634.513887ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XXX...] duration: 529.443342ms FAIL (2025-07-15 06:42:02 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 529.479973ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 429.008678ms FAIL (2025-07-15 06:41:20 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 429.043515ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 585.141459ms FAIL (2025-07-15 06:41:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 585.200936ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.029566669s FAIL (2025-07-15 06:41:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 1m0.029586434s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 7.740164868s FAIL (2025-07-15 06:41:07 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.74019008s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.041307178s FAIL (2025-07-15 06:40:58 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.041333385s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.031825928s FAIL (2025-07-15 06:40:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== FUVWYGP4SJOF6ZAFH3ZOWI3Q, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.162169468 (525h27m42.088464154s) [noise=true , address=79.127.205.233:4652 , piece_id=X7HW7X4IT7OCZ2EPFMJTETHF7B6KBYVD72ZQZYYOUC6WYBYFPCKQ, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.163951603 (525h27m42.090246298s) [noise=true , address=77.90.31.18:4822 , piece_id=O2RDAQRK2DHFEIYT567EB547QLEAIHN5GZCQL7JYKWUSA7V6JSTA, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.167457103 (525h27m42.093751792s) [noise=true , address=45.91.116.9:4293 , piece_id=JLVQVMPK32Z2KH66FR4YMU4GMLRF4JDMM7KOE5FFCBNURDGTSBTA, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.167835858 (525h27m42.094130907s) [noise=true , address=45.91.116.7:4027 , piece_id=EL3SYNYCXZWYLBGQTTR26RZNQV5UOBRQACUFF4XXAFFP6WLTKJTA, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.168189861 (525h27m42.094484547s) [noise=true , address=77.90.31.16:4758 , piece_id=HOFE6D6HQC4F6Q6ZELWUJYX2FEPIUGQMZGWHW2BLPEHUFNTALYWQ, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.168781718 (525h27m42.095076408s) [noise=true , address=77.90.31.21:4907 , piece_id=XCKVOOSFRK4SZHSPIO2UVONBSQUAN7K3GHBOMJEBOODM4364VV6A, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.168879683 (525h27m42.095174378s) [noise=true , address=77.90.31.17:4800 , piece_id=6D7VNHVPD7PP5BW7YW2M5PKVAT5ZALKXVQTVNVZ3ED4TUEDAES7A, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.169564226 (525h27m42.095858925s) [noise=true , address=77.90.31.14:4669 , piece_id=4HG3JYEUDJ5BY7NBHG66WRS6IGMDUX27BI37ZDTHUEYH7FWOSQ4A, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.170560106 (525h27m42.096854803s) [noise=true , address=77.90.31.10:4527 , piece_id=JRCG7QNYMFCHFRMU6NCWW76PCO5S2F3ANQKCJNHPARM5EWDIU4ZA, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.173546310 (525h27m42.099841012s) [noise=true , address=45.91.95.2:4018 , piece_id=H7DC6ZLMLW3JENBVGIMOTM6OMWYARMRRD5TNQLG7YOK6SOWWH4FQ, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.176132344 (525h27m42.10242704s) [noise=true , address=45.91.95.5:4028 , piece_id=GC33RVLHVWUEG5NC7FSKPOGPN55MQRPJ2ASB3265G2PRKNPT2QTA, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.178585420 (525h27m42.104880122s) [noise=true , address=156.146.43.225:4451 , piece_id=WVGCDYYHT2DW2AMBUITPYNCPGACQOJZPB62TYU5NSKW4CXT7LWTQ, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.181297378 (525h27m42.10759208s) [noise=true , address=45.91.116.4:4959 , piece_id=S66M77ZEOT6TRQI5KU7YRMZZ4LKON6RIOVPSF7O6IUSIHU2I4CFQ, seg_pos={0 0}, upload=420476]: Uploading piece... Jul 15 06:39:15.316719166 (525h27m42.243013846s) [upload=420476]: Next returned nil segment Jul 15 06:39:15.316739351 (525h27m42.243034027s) [upload=420476]: Waiting for error group managing segments... Jul 15 06:39:15.353539908 (525h27m42.279834601s) [noise=true , address=45.91.95.2:4018 , piece_id=H7DC6ZLMLW3JENBVGIMOTM6OMWYARMRRD5TNQLG7YOK6SOWWH4FQ, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:39:15.362698241 (525h27m42.28899295s) [noise=true , address=45.91.95.4:4006 , piece_id=NBTK5C7LHBR7UCDGDSDM6QBXP5EFT3LOYQHTTIO4MGCK74ORAYRA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:39:15.371205413 (525h27m42.297500093s) [noise=true , address=45.91.95.3:4002 , piece_id=7UB3C2CAWY6LXYFDPTGC4ZR4VD2JPB4PLDHD6NFAA2SN6QHKRLDA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:39:15.474791196 (525h27m42.401085876s) [noise=true , address=156.146.43.225:4451 , piece_id=WVGCDYYHT2DW2AMBUITPYNCPGACQOJZPB62TYU5NSKW4CXT7LWTQ, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:39:15.474853778 (525h27m42.40114845s) [noise=true , address=45.91.95.5:4028 , piece_id=GC33RVLHVWUEG5NC7FSKPOGPN55MQRPJ2ASB3265G2PRKNPT2QTA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:39:15.775760942 (525h27m42.702055628s) [noise=true , address=79.127.205.233:4652 , piece_id=X7HW7X4IT7OCZ2EPFMJTETHF7B6KBYVD72ZQZYYOUC6WYBYFPCKQ, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:39:23.074631944 (525h27m50.000926621s) [noise=true , address=79.127.205.234:4619 , piece_id=T4MKIUVGXHBLGNEJ6OGNSBCZLQV4FUVWYGP4SJOF6ZAFH3ZOWI3Q, seg_pos={0 0}, upload=420476]: Done uploading piece. err: <nil> Jul 15 06:40:15.083411415 (525h28m42.009706112s) [noise=true , address=77.90.31.15:4711 , piece_id=BGZPZ3UWPEWGPXCXUFJ6WBTAUOEVWRMP674A3R3ARYDVBJ3WUZVA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:12J2weU7Bmsr3RdQQ9QMuadT96XReBkyxTeaLJ2SEqVGGFX1PWu, address:77.90.31.15:4711): protocol: rpc: dial tcp 77.90.31.15:4711: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.083581520 (525h28m42.009876207s) [noise=true , address=77.90.31.18:4822 , piece_id=O2RDAQRK2DHFEIYT567EB547QLEAIHN5GZCQL7JYKWUSA7V6JSTA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:16EACrmREmTxBUPQKwsABtESUoZKWzcxKCknE81xK1nrDYpZSg, address:77.90.31.18:4822): protocol: rpc: dial tcp 77.90.31.18:4822: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.083737911 (525h28m42.010032584s) [noise=true , address=45.91.116.7:4027 , piece_id=EL3SYNYCXZWYLBGQTTR26RZNQV5UOBRQACUFF4XXAFFP6WLTKJTA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:12BQKHUMVYD7qDZW45LNEfgnkLVoSneDrQXViaMvcvAPdfmheMq, address:45.91.116.7:4027): protocol: write tcp 159.223.143.39:43518->45.91.116.7:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.083791091 (525h28m42.010085763s) [noise=true , address=77.90.31.10:4527 , piece_id=JRCG7QNYMFCHFRMU6NCWW76PCO5S2F3ANQKCJNHPARM5EWDIU4ZA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:1jEyiUUU9NCVFMU8frNggxTTYBMkehsqN4GLsFRkrxubdobGY2, address:77.90.31.10:4527): protocol: rpc: dial tcp 77.90.31.10:4527: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.083863295 (525h28m42.010157982s) [noise=true , address=77.90.31.16:4758 , piece_id=HOFE6D6HQC4F6Q6ZELWUJYX2FEPIUGQMZGWHW2BLPEHUFNTALYWQ, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:1ahTbggGRi6cJa4JPBfSEgLJYymrgVuLxSc15BV3qj5gPWPFiZ, address:77.90.31.16:4758): protocol: write tcp 159.223.143.39:49386->77.90.31.16:4758: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.083947323 (525h28m42.01024201s) [noise=true , address=77.90.31.17:4800 , piece_id=6D7VNHVPD7PP5BW7YW2M5PKVAT5ZALKXVQTVNVZ3ED4TUEDAES7A, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:1TxoPpye17ET1GR52xy2EmkUdq83x4T9TsfuwrKr8cYXf3HLYt, address:77.90.31.17:4800): protocol: write tcp 159.223.143.39:37714->77.90.31.17:4800: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.083957208 (525h28m42.010251884s) [noise=true , address=77.90.31.19:4873 , piece_id=KYXIYWUHZNJEF2VSRIOUZNSTOYGDE7CUGMVS47OOCU6TG4QM47LQ, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: write tcp 159.223.143.39:45338->77.90.31.19:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.084006117 (525h28m42.010300792s) [noise=true , address=45.91.116.9:4293 , piece_id=JLVQVMPK32Z2KH66FR4YMU4GMLRF4JDMM7KOE5FFCBNURDGTSBTA, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:1XMXDwZP5R7ZZGr8x57g3faasnxk7YJiLeBSEodBeCP3mnPuRG, address:45.91.116.9:4293): protocol: rpc: dial tcp 45.91.116.9:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.084062279 (525h28m42.010356966s) [noise=true , address=45.91.116.4:4959 , piece_id=S66M77ZEOT6TRQI5KU7YRMZZ4LKON6RIOVPSF7O6IUSIHU2I4CFQ, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:1RXPfJXrWzrCxzxRUk34LWLVuuwHHud41qVjn58qAwz7QiScSn, address:45.91.116.4:4959): protocol: rpc: dial tcp 45.91.116.4:4959: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.084133599 (525h28m42.010428286s) [noise=true , address=77.90.31.21:4907 , piece_id=XCKVOOSFRK4SZHSPIO2UVONBSQUAN7K3GHBOMJEBOODM4364VV6A, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:12RCGnjZC1UT9SUmFirNhS9pRBtxjcy91ss7c3Mk2kKnnHn2F6s, address:77.90.31.21:4907): protocol: write tcp 159.223.143.39:59452->77.90.31.21:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.084218863 (525h28m42.01051355s) [noise=true , address=77.90.31.14:4669 , piece_id=4HG3JYEUDJ5BY7NBHG66WRS6IGMDUX27BI37ZDTHUEYH7FWOSQ4A, seg_pos={0 0}, upload=420476]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: write tcp 159.223.143.39:58414->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:15.084267521 (525h28m42.010562214s) [seg_pos={0 0}, upload=420476]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 15 06:40:15.112513152 (525h28m42.038807842s) [upload=420476]: Done uploading segments... Jul 15 06:40:15.112543684 (525h28m42.038838377s) [upload=420476]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.031849054s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.036626584s FAIL (2025-07-15 06:40:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== ding piece... Jul 15 06:39:32.104980414 (525h27m59.03127511s) [noise=true , address=104.36.82.16:4293 , piece_id=EEWGJRTVDTLNPW6U4ANPRUHDL5CZL6OPAXSNEFMZNYWHWLI2M7IA, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.105584415 (525h27m59.031879111s) [noise=true , address=104.36.82.9:4005 , piece_id=ODLWFTO7YD3W5OG2TCKU2YL6UUDUX3MUK6KXBXKU7PJBO6ZS6FWA, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.105823220 (525h27m59.03211791s) [noise=true , address=45.91.116.11:4119 , piece_id=CUWHQLUKJXFZPKNUZYE3Z3QU2NLP44NPBMTBE35IZMI2NXUYNMWQ, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.106154991 (525h27m59.032449669s) [noise=true , address=45.91.116.11:4117 , piece_id=2QVKKPONLDPD4Z2PK6BZUHRU4C3FZBU5ANT7QP33TXRDAEAKDJ7A, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.106295424 (525h27m59.03259012s) [noise=true , address=45.91.116.5:4298 , piece_id=MUYPSGBGQPSS42Y5I6UJBIXSN5HCGQFR77GZFCRD7DRGW6SKXTFQ, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.106731966 (525h27m59.033026643s) [noise=true , address=77.90.31.19:4843 , piece_id=M5OENLX4UVFHSKVV77DNDJ55EXNQJIJE25IH2BN2LOWPAGEZQWDQ, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.106921561 (525h27m59.033216257s) [noise=true , address=77.90.31.16:4735 , piece_id=6D5KEZFM5TSZNY6SED6J42WVACE6EJZUREG4TC5TE42KWFPKLBOQ, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.107353364 (525h27m59.033648063s) [noise=true , address=77.90.31.18:4805 , piece_id=HPYPK7VT4MUHN55PV324EM3UFMVELALLFDP5TU4J6QNSJMRUHD2Q, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.107660904 (525h27m59.033955593s) [noise=true , address=45.91.116.6:4973 , piece_id=UDBSFWQF4SV4LGU62ARDZ5BVFFOAGEKZJVELT24PQJOOKG3T6VMQ, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.107848271 (525h27m59.034142966s) [noise=true , address=45.91.116.13:4155 , piece_id=2WOZ5EE63Z6KTTQQAHCUHMN4CKOQ3XHSHFXNOSEVXSF4TR54O64Q, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.108187987 (525h27m59.034482676s) [noise=true , address=104.36.82.13:4186 , piece_id=2UJAGFYUQEAHKT2VEDCGAKX2JKJAHMOUF3Z4U5FMY4HCFQAQL73A, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.108575378 (525h27m59.03487007s) [noise=true , address=77.90.31.18:4806 , piece_id=JVWZV5FWPUSYP5ZY6MM2UX3QTHKVGFTBWD73D6E4RT6ZUTYEM3PQ, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.108874555 (525h27m59.035169244s) [noise=true , address=45.91.116.10:4082 , piece_id=E4VOEEHW6QRXKPRBM7WPDPGWUHJOSCZT5TDV37YLCDTUY5FI6EVA, seg_pos={0 0}, upload=420480]: Uploading piece... Jul 15 06:39:32.185613400 (525h27m59.111908098s) [upload=420480]: Next returned nil segment Jul 15 06:39:32.185633353 (525h27m59.111928052s) [upload=420480]: Waiting for error group managing segments... Jul 15 06:39:32.194724020 (525h27m59.121018696s) [noise=true , address=104.36.82.13:4186 , piece_id=2UJAGFYUQEAHKT2VEDCGAKX2JKJAHMOUF3Z4U5FMY4HCFQAQL73A, seg_pos={0 0}, upload=420480]: Done uploading piece. err: <nil> Jul 15 06:39:32.195154753 (525h27m59.121449442s) [noise=true , address=104.36.82.9:4005 , piece_id=ODLWFTO7YD3W5OG2TCKU2YL6UUDUX3MUK6KXBXKU7PJBO6ZS6FWA, seg_pos={0 0}, upload=420480]: Done uploading piece. err: <nil> Jul 15 06:40:02.042340971 (525h28m28.96863567s) [noise=true , address=77.90.31.19:4843 , piece_id=M5OENLX4UVFHSKVV77DNDJ55EXNQJIJE25IH2BN2LOWPAGEZQWDQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.042603467 (525h28m28.968898166s) [noise=true , address=104.36.82.7:4910 , piece_id=VC4NJ62OMOCGCRHZ7Z3DCRDN3JMZAAECU75Q6PFR25D2XHNL4DIQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.042689720 (525h28m28.968984419s) [noise=true , address=104.36.82.16:4293 , piece_id=EEWGJRTVDTLNPW6U4ANPRUHDL5CZL6OPAXSNEFMZNYWHWLI2M7IA, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.042771318 (525h28m28.969066018s) [noise=true , address=45.91.116.5:4298 , piece_id=MUYPSGBGQPSS42Y5I6UJBIXSN5HCGQFR77GZFCRD7DRGW6SKXTFQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.042852760 (525h28m28.969147459s) [noise=true , address=45.91.116.7:4011 , piece_id=IAMHO43L4TQHUSNYB2LSXKW72Q4M44IHKCA27E2UA6LGCQ2QWDBA, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: rpc: dial tcp 45.91.116.7:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.044637834 (525h28m28.970932533s) [noise=true , address=45.91.116.10:4082 , piece_id=E4VOEEHW6QRXKPRBM7WPDPGWUHJOSCZT5TDV37YLCDTUY5FI6EVA, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.044725753 (525h28m28.971020443s) [noise=true , address=45.91.116.11:4117 , piece_id=2QVKKPONLDPD4Z2PK6BZUHRU4C3FZBU5ANT7QP33TXRDAEAKDJ7A, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.044918225 (525h28m28.971212917s) [noise=true , address=77.90.31.18:4805 , piece_id=HPYPK7VT4MUHN55PV324EM3UFMVELALLFDP5TU4J6QNSJMRUHD2Q, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.045168126 (525h28m28.971462824s) [noise=true , address=45.91.116.6:4973 , piece_id=UDBSFWQF4SV4LGU62ARDZ5BVFFOAGEKZJVELT24PQJOOKG3T6VMQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.045258569 (525h28m28.971553262s) [noise=true , address=45.91.116.13:4155 , piece_id=2WOZ5EE63Z6KTTQQAHCUHMN4CKOQ3XHSHFXNOSEVXSF4TR54O64Q, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.045343413 (525h28m28.971638104s) [noise=true , address=77.90.31.18:4806 , piece_id=JVWZV5FWPUSYP5ZY6MM2UX3QTHKVGFTBWD73D6E4RT6ZUTYEM3PQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 159.223.143.39:58736->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.045422372 (525h28m28.971717061s) [noise=true , address=45.91.116.11:4119 , piece_id=CUWHQLUKJXFZPKNUZYE3Z3QU2NLP44NPBMTBE35IZMI2NXUYNMWQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:43646->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.045481192 (525h28m28.971775885s) [noise=true , address=77.90.31.14:4663 , piece_id=TNUOX33QVN3KOQXEIB43W2XC4MUQ2Y7IDUGMKQWKKLPNCX7I6X2Q, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 159.223.143.39:36918->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:02.045658918 (525h28m28.971953609s) [noise=true , address=77.90.31.16:4735 , piece_id=6D5KEZFM5TSZNY6SED6J42WVACE6EJZUREG4TC5TE42KWFPKLBOQ, seg_pos={0 0}, upload=420480]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: context deadline exceeded; context deadline exceeded Jul 15 06:40:02.045685984 (525h28m28.971980684s) [seg_pos={0 0}, upload=420480]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jul 15 06:40:02.072724173 (525h28m28.999018863s) [upload=420480]: Done uploading segments... Jul 15 06:40:02.072756186 (525h28m28.999050882s) [upload=420480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 30.036648585s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.02658963s FAIL (2025-07-15 06:39:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== upload=420473]: Done begin upload segment. Jul 15 06:39:01.994091947 (525h27m28.920386645s) [noise=true , address=45.91.116.10:4082 , piece_id=RWOB7PBFJ7R7PATJAYYPPZXKD3MJVI37YGZ2O3CNGOTDWWODR3XA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.994100988 (525h27m28.920395687s) [noise=true , address=77.90.31.16:4734 , piece_id=VD723OCONXEE6BM4DC6KYQ23QWUHINHJFA5PQRLHXQFK2BW2YFFQ, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.994616402 (525h27m28.920911118s) [noise=true , address=104.36.82.15:4258 , piece_id=QORK3SBBIHENTXE3GOY5ZQ3MOQVMN7DIALZ4MBZ4C6FJA4WXEUWA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.995053242 (525h27m28.921347974s) [noise=true , address=45.91.116.4:4940 , piece_id=MSMPYT63ROMMBA2JHGTJORM5IOH3ZYQ4ZUB45DG6U5NI67MZV5JQ, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.995483218 (525h27m28.921777911s) [noise=true , address=104.36.82.9:4005 , piece_id=2ZI7YPDBVF3CQYQW2SUJWGN4HB5K3VGWEKAYK2XG2AY7RWM5TEUQ, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.995857020 (525h27m28.922151716s) [noise=true , address=104.36.82.14:4223 , piece_id=UMJLTUDS4VTN3KHOWERIYI2ESV6NSNTNHPGBD3CNGQ3CYGKLXZ7Q, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.996397002 (525h27m28.922691702s) [noise=true , address=104.36.82.13:4187 , piece_id=IXSPTKUPH3FI7BS5XFBMX2FINKEWEKPS5ZND24HNIGKCBPIJMAJA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.996806337 (525h27m28.923101034s) [noise=true , address=77.90.31.10:4517 , piece_id=AJRXVFK2OPKBSLCNJ2TCD32S6YBE7JF4RFCLKTCLZLWGCZ7AEOEA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.997203709 (525h27m28.92349841s) [noise=true , address=77.90.31.18:4806 , piece_id=TVZIIOZFOGKF65YQDPAKB6CYRGDFMIZ6O5UFPD4BSE2D5HFRESWA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.997644860 (525h27m28.923939554s) [noise=true , address=104.36.82.9:4007 , piece_id=YMN56QOSLEI6FPSVNBZRPDGCL5NCYO64JRFZVOZHDVRSVZROP6EA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.998042170 (525h27m28.924337102s) [noise=true , address=77.90.31.19:4842 , piece_id=ZQ5COOVTGRGHPLAGJMZY7KQYLKWJATVHFAIKCTAHC6D3I6AV5K5A, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.998441157 (525h27m28.924735854s) [noise=true , address=45.91.116.13:4153 , piece_id=HWUB43SNJCLXW2O4SCGU4PZ426YVCHODKK3HES7Z45YJO66N5WPQ, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.998828378 (525h27m28.925123074s) [noise=true , address=45.91.116.5:4298 , piece_id=OTCZ5TGPQ2SYA5PKVWMSUW243F2TOTIWNBBGNHYXX5PT6QILGOFQ, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.999271909 (525h27m28.925566609s) [noise=true , address=45.91.116.11:4118 , piece_id=63PC5CZACYDCRFQ4S4CYU4UE37UPLDH7NO5W2UJ6SHVFARISWUDQ, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:01.999657958 (525h27m28.925952635s) [noise=true , address=45.91.116.6:4975 , piece_id=GLSOXCQKNBRYSPUKJKEAUVPNCSRRXVEFJ7G427OEPLYB4RZ7IUIA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:02.001135498 (525h27m28.927430222s) [noise=true , address=77.90.31.11:4554 , piece_id=RV6FKOZ64DVRUTKKP5H3DOVKHSU2DQWCCMHQQUERM54RVYPZASUA, seg_pos={0 0}, upload=420473]: Uploading piece... Jul 15 06:39:02.106157975 (525h27m29.032452654s) [upload=420473]: Next returned nil segment Jul 15 06:39:02.106174810 (525h27m29.03246949s) [upload=420473]: Waiting for error group managing segments... Jul 15 06:39:02.113926628 (525h27m29.040221305s) [noise=true , address=104.36.82.9:4005 , piece_id=2ZI7YPDBVF3CQYQW2SUJWGN4HB5K3VGWEKAYK2XG2AY7RWM5TEUQ, seg_pos={0 0}, upload=420473]: Done uploading piece. err: <nil> Jul 15 06:39:02.114630403 (525h27m29.040925094s) [noise=true , address=104.36.82.9:4007 , piece_id=YMN56QOSLEI6FPSVNBZRPDGCL5NCYO64JRFZVOZHDVRSVZROP6EA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: <nil> Jul 15 06:39:02.115281036 (525h27m29.041575719s) [noise=true , address=104.36.82.15:4258 , piece_id=QORK3SBBIHENTXE3GOY5ZQ3MOQVMN7DIALZ4MBZ4C6FJA4WXEUWA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: <nil> Jul 15 06:39:02.115958982 (525h27m29.042253671s) [noise=true , address=104.36.82.14:4223 , piece_id=UMJLTUDS4VTN3KHOWERIYI2ESV6NSNTNHPGBD3CNGQ3CYGKLXZ7Q, seg_pos={0 0}, upload=420473]: Done uploading piece. err: <nil> Jul 15 06:39:02.116006571 (525h27m29.042301259s) [noise=true , address=104.36.82.13:4187 , piece_id=IXSPTKUPH3FI7BS5XFBMX2FINKEWEKPS5ZND24HNIGKCBPIJMAJA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: <nil> Jul 15 06:39:31.920002713 (525h27m58.846297386s) [noise=true , address=77.90.31.10:4517 , piece_id=AJRXVFK2OPKBSLCNJ2TCD32S6YBE7JF4RFCLKTCLZLWGCZ7AEOEA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 159.223.143.39:46176->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920058947 (525h27m58.846353633s) [noise=true , address=77.90.31.18:4806 , piece_id=TVZIIOZFOGKF65YQDPAKB6CYRGDFMIZ6O5UFPD4BSE2D5HFRESWA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920104700 (525h27m58.846399386s) [noise=true , address=45.91.116.11:4118 , piece_id=63PC5CZACYDCRFQ4S4CYU4UE37UPLDH7NO5W2UJ6SHVFARISWUDQ, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: rpc: dial tcp 45.91.116.11:4118: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920155822 (525h27m58.846450508s) [noise=true , address=45.91.116.6:4975 , piece_id=GLSOXCQKNBRYSPUKJKEAUVPNCSRRXVEFJ7G427OEPLYB4RZ7IUIA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920191396 (525h27m58.846486068s) [noise=true , address=45.91.116.10:4082 , piece_id=RWOB7PBFJ7R7PATJAYYPPZXKD3MJVI37YGZ2O3CNGOTDWWODR3XA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920254452 (525h27m58.846549138s) [noise=true , address=45.91.116.4:4940 , piece_id=MSMPYT63ROMMBA2JHGTJORM5IOH3ZYQ4ZUB45DG6U5NI67MZV5JQ, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920260283 (525h27m58.846554957s) [noise=true , address=45.91.116.5:4298 , piece_id=OTCZ5TGPQ2SYA5PKVWMSUW243F2TOTIWNBBGNHYXX5PT6QILGOFQ, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920298298 (525h27m58.846592983s) [noise=true , address=77.90.31.19:4842 , piece_id=ZQ5COOVTGRGHPLAGJMZY7KQYLKWJATVHFAIKCTAHC6D3I6AV5K5A, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920307496 (525h27m58.846602182s) [noise=true , address=77.90.31.11:4554 , piece_id=RV6FKOZ64DVRUTKKP5H3DOVKHSU2DQWCCMHQQUERM54RVYPZASUA, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920338204 (525h27m58.846632889s) [noise=true , address=45.91.116.13:4153 , piece_id=HWUB43SNJCLXW2O4SCGU4PZ426YVCHODKK3HES7Z45YJO66N5WPQ, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920369732 (525h27m58.846664404s) [noise=true , address=77.90.31.16:4734 , piece_id=VD723OCONXEE6BM4DC6KYQ23QWUHINHJFA5PQRLHXQFK2BW2YFFQ, seg_pos={0 0}, upload=420473]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37894->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:39:31.920386274 (525h27m58.846680946s) [seg_pos={0 0}, upload=420473]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 15 06:39:31.944731824 (525h27m58.871026504s) [upload=420473]: Done uploading segments... Jul 15 06:39:31.944755918 (525h27m58.871050607s) [upload=420473]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.026618661s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 378.684198ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 06:03:05 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 378.702545ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 369.690199ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 06:02:11 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 369.718002ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 440.658554ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 06:01:07 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 440.693684ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 589.154364ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 06:00:13 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 589.178028ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 444.438254ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:59:27 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 444.470161ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 578.605966ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:58:59 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 578.632315ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 706.992442ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:58:12 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 707.022249ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 519.094706ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:57:38 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 519.123166ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 474.8733ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:56:47 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 474.901429ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 346.795058ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:55:49 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 346.822827ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 354.803629ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:55:10 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 354.830809ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 642.994061ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:54:21 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 643.023902ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 357.618715ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:53:16 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 357.647775ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 447.980096ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:52:25 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 448.020067ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 593.353394ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:51:28 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 593.381622ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 484.179097ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:50:41 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 484.208143ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 407.480564ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:50:09 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 407.504721ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 563.196105ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:49:48 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 563.234502ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 363.014783ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:48:54 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 363.061464ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 391.935681ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Date=Tue, 15 Jul 2025 05:17:16 GMT Content-Length=11 Content-Type=text/plain; charset=utf-8 target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-15 05:48:13 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "Bad Gateway" Duration: 391.963439ms edge ipfs us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)