Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration: 451.161285ms FAIL (2025-07-15 07:10:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 451.193549ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 557.805294ms FAIL (2025-07-15 07:10:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 557.896055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............XXX..........................................] duration: 455.534706ms FAIL (2025-07-15 07:09:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 455.563683ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 540.74273ms FAIL (2025-07-15 07:09:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 540.77876ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................XXX.........................................] duration: 469.450812ms FAIL (2025-07-15 07:09:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 469.479124ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 648.650249ms FAIL (2025-07-15 07:08:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 648.70808ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................XXX........................................] duration: 422.937949ms FAIL (2025-07-15 07:08:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 422.968986ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 701.956691ms FAIL (2025-07-15 07:07:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 702.001901ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................XXX.......................................] duration: 507.013797ms FAIL (2025-07-15 07:07:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 507.040275ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................XXX......................................] duration: 464.135829ms FAIL (2025-07-15 07:06:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 464.240202ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 602.299134ms FAIL (2025-07-15 07:06:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 602.400115ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................XXX.....................................] duration: 541.715944ms FAIL (2025-07-15 07:05:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 541.754956ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 709.734411ms FAIL (2025-07-15 07:05:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 709.767049ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................XXX....................................] duration: 640.140914ms FAIL (2025-07-15 07:05:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 640.173588ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 502.927774ms FAIL (2025-07-15 07:04:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 502.96066ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................XXX...................................] duration: 430.178226ms FAIL (2025-07-15 07:04:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 430.218206ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................XXX..................................] duration: 432.789564ms FAIL (2025-07-15 07:03:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 432.8233ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 616.539669ms FAIL (2025-07-15 07:03:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 616.57194ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................XXX.................................] duration: 478.045937ms FAIL (2025-07-15 07:03:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 478.082175ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 505.405393ms FAIL (2025-07-15 07:02:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 505.433423ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 1.061399122s FAIL (2025-07-15 07:02:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 1.061430201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................XXX................................] duration: 654.390621ms FAIL (2025-07-15 07:01:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 654.42208ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 734.983723ms FAIL (2025-07-15 07:01:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 735.016297ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................XXX...............................] duration: 489.63787ms FAIL (2025-07-15 07:01:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 489.697061ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration: 469.483223ms FAIL (2025-07-15 07:00:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 469.517401ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................XXX.............................] duration: 557.652463ms FAIL (2025-07-15 07:00:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 557.692374ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 2.09095829s FAIL (2025-07-15 07:00:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 2.091001004s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................XXX............................] duration: 691.248039ms FAIL (2025-07-15 07:00:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 691.290308ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................XXX...........................] duration: 407.430685ms FAIL (2025-07-15 06:59:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 407.458519ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 670.168996ms FAIL (2025-07-15 06:59:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 670.19168ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................XXX..........................] duration: 410.421614ms FAIL (2025-07-15 06:58:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 410.47624ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................XXX.........................] duration: 390.930714ms FAIL (2025-07-15 06:58:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 390.965049ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 530.029694ms FAIL (2025-07-15 06:58:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 530.062104ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................XXX........................] duration: 446.534153ms FAIL (2025-07-15 06:57:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 446.565609ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 450.841888ms FAIL (2025-07-15 06:57:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 450.90439ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration: 498.570765ms FAIL (2025-07-15 06:57:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 498.603839ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 587.079459ms FAIL (2025-07-15 06:56:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 587.109727ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................XXX......................] duration: 462.066613ms FAIL (2025-07-15 06:56:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 462.155717ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 586.74018ms FAIL (2025-07-15 06:55:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 586.769456ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 738.145636ms FAIL (2025-07-15 06:55:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 738.183458ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration: 429.74303ms FAIL (2025-07-15 06:55:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 429.775976ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 748.68559ms FAIL (2025-07-15 06:54:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 748.717763ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................XXX....................] duration: 631.681325ms FAIL (2025-07-15 06:54:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 631.727456ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 518.151028ms FAIL (2025-07-15 06:53:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 518.204456ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................XXX...................] duration: 463.529443ms FAIL (2025-07-15 06:53:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 463.585293ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 438.350432ms FAIL (2025-07-15 06:53:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 438.381749ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XXX..................] duration: 400.374337ms FAIL (2025-07-15 06:52:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 400.408929ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................XXX.................] duration: 427.675028ms FAIL (2025-07-15 06:52:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.841067446 (525h28m55.767362136s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842190686 (525h28m55.768485372s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.842741155 (525h28m55.769035845s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.843558552 (525h28m55.769853244s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845412066 (525h28m55.771706757s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.845779870 (525h28m55.772074547s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846101607 (525h28m55.772396299s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846410882 (525h28m55.772705578s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.846856835 (525h28m55.773151529s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847162658 (525h28m55.773457331s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.847507116 (525h28m55.773801802s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848161715 (525h28m55.774456411s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848270353 (525h28m55.774565043s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.848668674 (525h28m55.774963371s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Uploading piece... Jul 15 06:40:28.926022552 (525h28m55.852317233s) [upload=420493]: Next returned nil segment Jul 15 06:40:28.926041073 (525h28m55.85233575s) [upload=420493]: Waiting for error group managing segments... Jul 15 06:40:28.934634958 (525h28m55.860929636s) [noise=true , address=104.36.82.6:4873 , piece_id=QZRTL4VTFNUUHWQUUBD6X5V3CDA2DJVCINW6JXK6ZX4CKGRBRXXQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.934957930 (525h28m55.861252619s) [noise=true , address=104.36.82.16:4294 , piece_id=7I6OUGLL2JF36LSL6Q3YJTF323E5UJJYA7P7YK45ZRHQRXXEDL4Q, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935286303 (525h28m55.861580978s) [noise=true , address=104.36.82.15:4258 , piece_id=JLH4QJBQ5QFILAAINFXBALPYVJOXDVDBTGBNCJSKIM6STVBYFBUQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:28.935990357 (525h28m55.862285037s) [noise=true , address=104.36.82.5:4838 , piece_id=BA57HGVNQYVIJGJA5RRU4QKCBWKDKLUDS36ETMGO7RCCE4C4I7ZQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: <nil> Jul 15 06:40:58.776020327 (525h29m25.702315013s) [noise=true , address=77.90.31.10:4517 , piece_id=JTGFACMVXTKCZC2BPMLEXRMGE6ESRWG54C6JCNBHQPHDLLR3V2QA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776074950 (525h29m25.702369629s) [noise=true , address=45.91.116.6:4973 , piece_id=U7YJAIBH6BE3UVD43QUSL4BOOF5DK2I5LF5EJPPPPJZ5UK7H6PPA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776121344 (525h29m25.702416024s) [noise=true , address=77.90.31.17:4772 , piece_id=DPPLGRJ2DFWMYNBESQRTSTING6UOAPJXRAR7DHG5DDB6LRQ5AVCQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776175296 (525h29m25.702469983s) [noise=true , address=45.91.116.7:4010 , piece_id=BYYUAAIHZRRP4CP5KSHEIQQK5M5PJ5J7NAEVOLH2GSK6UBUTGHGQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776197837 (525h29m25.702492512s) [noise=true , address=77.90.31.10:4518 , piece_id=QVOYK7MQEGL77P6BSD7DFZX7XWOJYHUUC3MHXN2PEWZVDYBWAHNQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: rpc: dial tcp 77.90.31.10:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776227152 (525h29m25.702521829s) [noise=true , address=45.91.116.13:4155 , piece_id=VZ2XEQVSC2B27WGXYUQXFTNHSL2VWSRDYETXVS4XKOBYOTEFP2HA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 159.223.143.39:48174->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776260695 (525h29m25.70255537s) [noise=true , address=45.91.116.8:4045 , piece_id=CG7XH3Z3RSRB75NDY726OX2HG52YYRYYKLJ4KZ3XS45H7MLZKJQQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776274299 (525h29m25.702568975s) [noise=true , address=45.91.116.10:4083 , piece_id=AB4C3APZLCTEJY2IRT63G4UYRC3FHBJ7V7O2XUD3KTK5POL5UERQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 159.223.143.39:40206->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776321523 (525h29m25.70261621s) [noise=true , address=77.90.31.11:4554 , piece_id=VOCQQPTONV7TKI2XUSBKZ76Q7PAWRN22R7WOA5FNKPMJ4ETQJURQ, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776330301 (525h29m25.702624977s) [noise=true , address=45.91.116.4:4940 , piece_id=LS6OIIJAWSKEHHDNYOPT4L25KTZZGJTUXQWS4EQXYWYS4KS62BSA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 159.223.143.39:35896->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776339431 (525h29m25.702634109s) [noise=true , address=77.90.31.17:4771 , piece_id=NX3KG2JBDP2AXF3W6ETQ4AZURYE42BRTWO3NGBLOWVOCASE2RXUA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 159.223.143.39:41706->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776368787 (525h29m25.702663466s) [noise=true , address=77.90.31.15:4698 , piece_id=TWXJDOQ6DPLN6RMBIR453TSXBAODGKBNLSI6JMYTGFB7D3QCIMGA, seg_pos={0 0}, upload=420493]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:40:58.776408059 (525h29m25.702702736s) [seg_pos={0 0}, upload=420493]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 15 06:40:58.814777453 (525h29m25.741072132s) [upload=420493]: Done uploading segments... Jul 15 06:40:58.814808768 (525h29m25.741103453s) [upload=420493]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 427.718658ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 687.209404ms FAIL (2025-07-15 06:51:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 687.24306ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 538.272966ms FAIL (2025-07-15 06:51:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece... Jul 15 06:40:15.278581034 (525h28m42.20487574s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279073844 (525h28m42.205368539s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.279619815 (525h28m42.205914504s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.280582440 (525h28m42.206877135s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.281735009 (525h28m42.208029701s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.282567902 (525h28m42.208862591s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.283283854 (525h28m42.209578546s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.285562645 (525h28m42.211857325s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.286257756 (525h28m42.212552436s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.287131879 (525h28m42.213426559s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Uploading piece... Jul 15 06:40:15.472371743 (525h28m42.398666434s) [upload=420490]: Next returned nil segment Jul 15 06:40:15.472390981 (525h28m42.398685679s) [upload=420490]: Waiting for error group managing segments... Jul 15 06:40:15.504471241 (525h28m42.430765935s) [noise=true , address=45.91.95.3:4032 , piece_id=OA5WEOHHDCKXEDKJQA2O5JJZ743CQCZO6WLFBXO4GCYLR663JQ3A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.514931454 (525h28m42.441226144s) [noise=true , address=45.91.95.5:4014 , piece_id=YFYBUHNQNILH66IE37SZFGFMGD5JZA6GJGWMDFR67EMFLBISZAFQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.517626455 (525h28m42.443921147s) [noise=true , address=45.91.95.4:4023 , piece_id=YKSPRXMWXIGIFL4XGG7BVGFYM7KIQTX54IMQAICZFY5XDIIU37FQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.613544385 (525h28m42.539839077s) [noise=true , address=79.127.201.214:4324 , piece_id=OKCUROKMQMKEGXY23VPYVFDFJYJJVL25VWM7LHIQ4MJE3WBVNVMA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:40:15.620924601 (525h28m42.547219294s) [noise=true , address=79.127.201.215:4359 , piece_id=3PP5PB3FMALB6D5S2FJFCXZTER4CM3NMJZYBJWAUNEYNG5T6Q64A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: <nil> Jul 15 06:41:15.201243831 (525h29m42.127538524s) [noise=true , address=45.91.116.10:4110 , piece_id=OYECJBO3RGGMKSDNY5U25PHS6DPJ6YPTYQ76E2TGFATQWTJREALA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:123KwusATu1ZKtNNVHZZZBE67ZEnVb8puyfQidBi3fEWREqdf1h, address:45.91.116.10:4110): protocol: rpc: dial tcp 45.91.116.10:4110: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201491982 (525h29m42.127786671s) [noise=true , address=45.91.116.13:4181 , piece_id=5EAC7PXRV3Y6ZT34RG37UEMWCOAYKWT6BRSTRX42XQLVA5F3YNNQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: write tcp 159.223.143.39:45464->45.91.116.13:4181: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201635580 (525h29m42.127930267s) [noise=true , address=45.91.116.6:4994 , piece_id=HYMLTCWURZG6PNASGSCCSRQXTWGPHBKHI2QQAU5BOFB5F7OPD7FA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:13bDFHbqDxbPr7n4PTQYCXE151sUUFPkidaY1KBZSPcPBBAnFo, address:45.91.116.6:4994): protocol: write tcp 159.223.143.39:54454->45.91.116.6:4994: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201726830 (525h29m42.128021506s) [noise=true , address=77.90.31.17:4774 , piece_id=HBDZMU57RV4LARCNJT6X72Z4ZBNSVKHENI24TS2UWWTN4KZC7VIQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 159.223.143.39:45776->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.201780346 (525h29m42.128075024s) [noise=true , address=77.90.31.15:4712 , piece_id=YHPTVLP5KUDPZMLDZYG46IHAH5PZ75KNLQ6Z6CRTSHKIEPPAA46A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 159.223.143.39:44882->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202002741 (525h29m42.128297432s) [noise=true , address=77.90.31.19:4847 , piece_id=L6HXKSLFKN5GZN7ANGT6K5SKSMBCAJNMFXBICX7YC5C2J7DFL7AQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12RkRJoWfBB5tUsZScQgydJhY5XffVuQn1JUa1ZHThjcP3bUfJv, address:77.90.31.19:4847): protocol: write tcp 159.223.143.39:60486->77.90.31.19:4847: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202083448 (525h29m42.128378129s) [noise=true , address=77.90.31.21:4901 , piece_id=V3LPDPV3TNCTJCCONYOIBJP5Q5O62LNXOUNKQR2U3KNBEJX3JUGA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12vnJeNLPrxvFyZMsfC61nZB1t3n77x9qgmjrYAhH6wyc7KfC9U, address:77.90.31.21:4901): protocol: write tcp 159.223.143.39:32996->77.90.31.21:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202127457 (525h29m42.128422144s) [noise=true , address=77.90.31.11:4580 , piece_id=KTWZK2NJR6ESLE4CEZJD64COSWIQK7WX4YRXDLLUKWJMIILZROKA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D, address:77.90.31.11:4580): protocol: write tcp 159.223.143.39:36244->77.90.31.11:4580: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202160777 (525h29m42.128455449s) [noise=true , address=77.90.31.16:4754 , piece_id=OYGSOOVEPIJC3ZL6PRAHZSTL7BKPFWAD6U5FT2YLAJETFXYAGNXQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:54514->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202229558 (525h29m42.128524246s) [noise=true , address=77.90.31.13:4630 , piece_id=MPL6CSLWUNWA42VSROSY4QVVMPJL7JVI7JMQ2HRS72VCM52AX62A, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1gyQJLnzWmRESyd8m6G7aZWsQwVfqRQNARxRmd7GwwfYwhuzVn, address:77.90.31.13:4630): protocol: write tcp 159.223.143.39:58386->77.90.31.13:4630: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202400569 (525h29m42.128695244s) [noise=true , address=77.90.31.17:4778 , piece_id=AC6F36V262HEIQCMHZX7ZB3DYBEKWJXAIPNPCX4V4H2MNRV4VIQQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:12icjsyKdwPqDddPbEZpCJvX1pxmTLSy8ZfFUMCJU1SxPsKGdZo, address:77.90.31.17:4778): protocol: write tcp 159.223.143.39:42642->77.90.31.17:4778: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202503700 (525h29m42.128798376s) [noise=true , address=45.91.116.14:4212 , piece_id=ST2TKG6QARKQMRYKI2XKQAWNL6KO2FPSJ7GNZ5EWVRX2ZR5VW2IQ, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 159.223.143.39:39712->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202558148 (525h29m42.128852826s) [noise=true , address=77.90.31.18:4838 , piece_id=REYD24TR7ROA272SDBKIIETP7N2MDFX3G74X5QWSZ25MZMCYSNDA, seg_pos={0 0}, upload=420490]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: write tcp 159.223.143.39:46310->77.90.31.18:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 15 06:41:15.202592484 (525h29m42.128887163s) [seg_pos={0 0}, upload=420490]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 15 06:41:15.227598864 (525h29m42.153893542s) [upload=420490]: Done uploading segments... Jul 15 06:41:15.227623232 (525h29m42.15391792s) [upload=420490]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 538.313677ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)