Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 717.117927ms FAIL (2025-07-29 09:14:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 717.234105ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 536.799328ms FAIL (2025-07-29 09:13:40 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 536.826659ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 665.524115ms FAIL (2025-07-29 09:13:23 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 665.559191ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 493.142567ms FAIL (2025-07-29 09:12:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 493.175614ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 497.965807ms FAIL (2025-07-29 09:12:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 498.000515ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 535.62681ms FAIL (2025-07-29 09:12:24 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 535.659648ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 546.70163ms FAIL (2025-07-29 09:11:40 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 546.736095ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 1.093597945s FAIL (2025-07-29 09:11:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1.093628722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 740.395966ms FAIL (2025-07-29 09:10:58 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 740.436317ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 511.276143ms FAIL (2025-07-29 09:10:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 511.304448ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.033219957s FAIL (2025-07-29 09:10:09 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1.033335896s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 1.012975064s FAIL (2025-07-29 09:09:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1.013005879s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.278333122s FAIL (2025-07-29 09:09:03 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1.278363979s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 668.296237ms FAIL (2025-07-29 09:08:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 668.319782ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 628.211422ms FAIL (2025-07-29 09:08:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 628.242462ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 451.623159ms FAIL (2025-07-29 09:07:49 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 451.657656ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 531.125842ms FAIL (2025-07-29 09:07:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 531.191417ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 533.599022ms FAIL (2025-07-29 09:06:47 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 533.642785ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 610.645834ms FAIL (2025-07-29 09:06:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 610.698682ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 529.216733ms FAIL (2025-07-29 09:06:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 529.35636ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 482.370129ms FAIL (2025-07-29 09:05:47 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 482.44423ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 633.055997ms FAIL (2025-07-29 09:05:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 633.084806ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 560.331711ms FAIL (2025-07-29 09:04:49 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 560.377015ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 479.795084ms FAIL (2025-07-29 09:04:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 479.823816ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.3363015s FAIL (2025-07-29 09:03:42 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1.336346416s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 540.394915ms FAIL (2025-07-29 09:03:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 540.436975ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 482.576098ms FAIL (2025-07-29 09:03:15 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 482.613191ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 477.263245ms FAIL (2025-07-29 09:02:47 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 477.293926ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 574.187716ms FAIL (2025-07-29 09:02:21 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 574.221536ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 519.127583ms FAIL (2025-07-29 09:02:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 519.159229ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 499.715428ms FAIL (2025-07-29 09:02:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 499.816112ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 601.413307ms FAIL (2025-07-29 09:01:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 601.446023ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 479.01161ms FAIL (2025-07-29 09:01:24 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 479.045679ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 716.255525ms FAIL (2025-07-29 09:01:03 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 716.299237ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 589.940284ms FAIL (2025-07-29 09:00:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 589.998906ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 620.213547ms FAIL (2025-07-29 09:00:31 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 620.23998ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 428.625279ms FAIL (2025-07-29 08:59:55 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 428.714856ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 483.604758ms FAIL (2025-07-29 08:59:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 483.672494ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 455.961118ms FAIL (2025-07-29 08:59:20 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 455.99254ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 561.288423ms FAIL (2025-07-29 08:59:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 561.321296ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 380.265413ms FAIL (2025-07-29 08:59:01 UTC) orbiter warning: uplink: too many requests Duration: 380.297095ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 666.146152ms FAIL (2025-07-29 08:58:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 666.214242ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 533.177251ms FAIL (2025-07-29 08:58:37 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 533.212914ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 178.504658ms FAIL (2025-07-29 08:58:30 UTC) orbiter warning: uplink: too many requests Duration: 178.618233ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 448.979213ms FAIL (2025-07-29 08:58:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 449.009133ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 466.005341ms FAIL (2025-07-29 08:58:10 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 11) Duration: 466.033147ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 225.891281ms FAIL (2025-07-29 08:57:58 UTC) orbiter warning: uplink: too many requests Duration: 225.915278ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 406.950177ms FAIL (2025-07-29 08:57:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 406.977291ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 219.598595ms FAIL (2025-07-29 08:57:31 UTC) orbiter warning: uplink: too many requests Duration: 219.626667ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 401.116058ms FAIL (2025-07-29 08:57:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 29396822s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.508952596 (15h48m15.329903666s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509401082 (15h48m15.330352153s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.509818482 (15h48m15.330769553s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510225954 (15h48m15.331177025s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.510633841 (15h48m15.331584912s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511255635 (15h48m15.332206685s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.511671340 (15h48m15.332622423s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512081651 (15h48m15.333032736s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512527046 (15h48m15.333478116s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.512949295 (15h48m15.333900368s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513325303 (15h48m15.334276369s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.513697448 (15h48m15.334648518s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514086620 (15h48m15.335037686s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.514481567 (15h48m15.335432639s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517240884 (15h48m15.338191952s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517683523 (15h48m15.338634588s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.517994621 (15h48m15.338945683s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Uploading piece... Jul 29 08:33:28.637090704 (15h48m15.45804177s) [upload=15205]: Next returned nil segment Jul 29 08:33:28.637111137 (15h48m15.458062204s) [upload=15205]: Waiting for error group managing segments... Jul 29 08:33:28.644180102 (15h48m15.465131163s) [noise=true , address=104.36.82.54:4180 , piece_id=VC7B62RZ7VBXVQOP37CYRVWCYBP32YVI7OTC5XVD2B2ERRGR6PRA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645592621 (15h48m15.466543676s) [noise=true , address=104.36.82.44:4875 , piece_id=57S6ZDPCLD4TQAHTJXFQPT6QIBJPS4TBX2ZB5N75JLNXNTOAML6A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.645978711 (15h48m15.466929759s) [noise=true , address=104.36.82.47:4542 , piece_id=GBSP4F7KD7JDGMMFUU225YJJOZNSYN3R5GFSM5YTCEX5KU7C4ZIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646138005 (15h48m15.467089069s) [noise=true , address=104.36.82.46:4484 , piece_id=Q5GI6NHHARUAW2UIH3C2XXH4CATE7JAPDG3MMYG5MVRNH4MVU5QA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.646644108 (15h48m15.467595157s) [noise=true , address=104.36.82.3:4770 , piece_id=3L2ES5H7QZXYQY4M3TLTACXXGENEEIYNK5XSEZQ4LQ22ARV2ORIQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647053901 (15h48m15.468004958s) [noise=true , address=104.36.82.47:4540 , piece_id=IWWXSY6BF7VTLHERVVFPUBR5KNFXAZMVNBRQ5BMACGGGEMZUUJMQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647286518 (15h48m15.468237565s) [noise=true , address=104.36.82.46:4498 , piece_id=3M6OZG632UKHMQKYIMAREZ6GKCL4GWUANWLS3C5FJ3Q3Z2GUK3YA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647762434 (15h48m15.468713495s) [noise=true , address=104.36.82.12:4173 , piece_id=H6CRX6F7HLEOEHCJEGODL5TARQWTMR67TJFBXPHCIXI4OZG4B2CA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.647881559 (15h48m15.468832609s) [noise=true , address=104.36.82.4:4816 , piece_id=IES5GV3FHWWRUJESJJKK5PXGSTEITI3NDJOEWKYYSJB2CPE2ZCAQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:33:28.649052927 (15h48m15.470003986s) [noise=true , address=104.36.82.53:4129 , piece_id=JLQVER4QJXVNBWL3245NODYTH3PCJC62PK5XZOR2HHHINXPHXRZQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: <nil> Jul 29 08:34:28.398854510 (15h49m15.219805573s) [noise=true , address=104.36.82.15:4287 , piece_id=KPL4LQW6VOXONTMT5O5L56KC32UF5XW574GEPVKGBRH4LPDUYLHA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12GvK5gHFVFqvbPjGEKssvcjczZYCw1TVf1q3y7G9qhhhgeU6BJ, address:104.36.82.15:4287): protocol: rpc: dial tcp 104.36.82.15:4287: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.398948736 (15h49m15.219899797s) [noise=true , address=104.36.82.15:4270 , piece_id=JOCDSZIED633CCYHWGSHYOWN6ENN6UMWA66T4OPLIBT6G4M3F2RA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1nkuzy3vxeLdHhVs73Nceje7qXDiGJ1zEhpGgGKSk3cKb7vJbB, address:104.36.82.15:4270): protocol: rpc: dial tcp 104.36.82.15:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399236812 (15h49m15.220187859s) [noise=true , address=104.36.82.10:4094 , piece_id=CTP6DS7PU64SF7RIRGU5K2ZDVNZWHJSGDD2TQPDPDKURN4IE3RYA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: write tcp 159.223.143.39:47998->104.36.82.10:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399307635 (15h49m15.220258695s) [noise=true , address=104.36.82.5:4847 , piece_id=ECZM2XPPNMWPZHALAFSEGKCA2MHIE5RLCODT4YKW4VWPV3U3CRWQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:104.36.82.5:4847): protocol: rpc: dial tcp 104.36.82.5:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399531548 (15h49m15.220482591s) [noise=true , address=104.36.82.55:4199 , piece_id=34CQP7ADVD7TOMVG6H7KFIIDIRJ2MKFSY7JWOUUUBYPZMSBPG6MQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399623256 (15h49m15.220574315s) [noise=true , address=104.36.82.42:4831 , piece_id=X4QAWE6B5437AAMLFF4ZSNGJ3OLCHZ4SV3MF3X2FXIC5AKQIMOEA, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12hZK8xSEF54FaK2tBkB4BVHywDuJs45HisTs2SJ7ZrnuKkoDDX, address:104.36.82.42:4831): protocol: rpc: dial tcp 104.36.82.42:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399684285 (15h49m15.220635344s) [noise=true , address=104.36.82.6:4892 , piece_id=AU2UKHKRQWF2VPJ5QVAU2GY463JUNYPCDTHZJVHHM5RVWW2UT4QQ, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:12MmZivyjXMMHcDgU3aG4HkWqBgY2QRf4Q1pkxJDF5QKSTQ7RXA, address:104.36.82.6:4892): protocol: rpc: dial tcp 104.36.82.6:4892: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399697381 (15h49m15.220648427s) [noise=true , address=104.36.82.13:4189 , piece_id=PMGLKJR4AGWV4YXFEAZD3KFZL73HTD6YPFM2XJCVHF6RJHD4E37A, seg_pos={0 0}, upload=15205]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: rpc: dial tcp 104.36.82.13:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 08:34:28.399729611 (15h49m15.220680656s) [seg_pos={0 0}, upload=15205]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 29 08:34:28.428326033 (15h49m15.24927709s) [upload=15205]: Done uploading segments... Jul 29 08:34:28.428418415 (15h49m15.249369478s) [upload=15205]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 401.157231ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)