Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 18.914771787s FAIL (2025-06-22 00:07:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 05:59.086456248 (79h42m34.649779236s) [upload=63806]: Upload finished. err: <nil> Jun 22 00:06:35.070905470 (79h43m10.634228474s) [upload=63815]: Starting upload Jun 22 00:06:35.071010688 (79h43m10.63433369s) [upload=63815]: Done starting upload Jun 22 00:06:35.071201792 (79h43m10.634524782s) [upload=63815]: Uploading segments... Jun 22 00:06:35.071643829 (79h43m10.634966841s) [upload=63815]: Got next segment. Inline: false Jun 22 00:06:35.071764824 (79h43m10.63508783s) [upload=63815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 22 00:06:35.071774624 (79h43m10.635097634s) [upload=63815]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 22 00:06:35.107204892 (79h43m10.670527901s) [seg_pos={0 0}, upload=63815]: Begin upload segment... Jun 22 00:06:35.107425462 (79h43m10.670748472s) [noise=true , address=104.36.82.42:6803 , piece_id=DZ5E65IJ34B66MYV7MYQ4TIJE44PWIWAKGYRCVDPYGTLKF6UKN4A, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.107974355 (79h43m10.671297357s) [noise=true , address=77.90.31.14:4692 , piece_id=O3GBKLZWYWSUKTZONTAFH3GPZFGTQTRQELSCMT6KB6LZFKETGKVQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.108920911 (79h43m10.672243923s) [noise=true , address=104.36.82.21:6493 , piece_id=GGBVGKVSAWFYTLYPNME4EBY3QAMQFA2MESSDNWAG7WLAD3NCPHKA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.109421565 (79h43m10.672744573s) [noise=true , address=104.36.82.20:6468 , piece_id=CCSQEOUXRJC7AXDSH4EASAPPSK7365ZA72OZI56C4NTVL44U46WQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.109844294 (79h43m10.673167296s) [noise=true , address=79.127.205.234:5611 , piece_id=6NNVTKVHHZVN6BI34CVNW5DPLE7HI4F52B3TMASMKWKRM43D4QVQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.110267970 (79h43m10.673590974s) [noise=true , address=79.127.219.38:4036 , piece_id=SSRGW7SWOVNO4XSRIKLNDXW5F2R46FTOFQJNUAQQ34PILKHX3I4A, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.111464573 (79h43m10.674787575s) [seg_pos={0 0}, upload=63815]: Done begin upload segment. Jun 22 00:06:35.111520150 (79h43m10.674843159s) [noise=true , address=77.90.31.17:4803 , piece_id=MA4QUVSB5MBKWVHZSU2HTSD7HNKFCDM5LH6AOMW2JG2VJAUPNWJQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.112035917 (79h43m10.675358922s) [noise=true , address=79.127.220.98:4402 , piece_id=TYEZTCGQRZTBHMZZDDZQTNG4DZCTFIJ3REAFDSOQWZW3UHWKSJTA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.112494883 (79h43m10.675817888s) [noise=true , address=104.36.82.55:4204 , piece_id=TII6GSL5AQ65NVLWSTR4FV4Y3KIGOZTH6VKYSFVK3SNAQ4VAM4HA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.112954016 (79h43m10.676277025s) [noise=true , address=104.36.82.13:6196 , piece_id=IGOXQ3NJ5IDXUMUDQS7LVSSFVLZHHDL4ZDUQRCRT3SGDP6C2WBFA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.113391990 (79h43m10.676714991s) [noise=true , address=104.36.82.53:4146 , piece_id=3R7U2WZVOFQK3YAMMBQFB62JEV4GO72YSSKZHGWUCOLGSHPYWVEQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.113834067 (79h43m10.677157075s) [noise=true , address=104.36.82.5:5848 , piece_id=RESM36SSKLRSRBTHF24KRJYXRLVJ2EGYLZYMOMAERDSBHQNGZHTQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.114273706 (79h43m10.677596718s) [noise=true , address=104.36.82.52:4108 , piece_id=CK6YCPRZFTVJNOBD6WJGXIFXJ3P5ZTKGK76CTN4HGJ3GWVYM6XPQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.114737587 (79h43m10.678060595s) [noise=true , address=77.90.31.21:4887 , piece_id=OOVHZJWHOBHJUBWPZBJ7JFHAVCONGWOIQKNSWRUDRLMUVDJVW3JA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.123351328 (79h43m10.68667432s) [noise=true , address=104.36.82.54:4186 , piece_id=UWTOIEDDR76OM6Y2ACCRIUTAPMMMR7MDZZJ4WKWR5DJ55IQQ67QA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.124034043 (79h43m10.68735705s) [noise=true , address=104.36.82.56:4241 , piece_id=YJESR3E5OCIDCHNDKB4EO5MPKH5IP4WIWP5FBLVF7A4SSWIDMPDQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.124489284 (79h43m10.687812288s) [noise=true , address=77.90.31.18:4808 , piece_id=FIJJMZP53YXE7OJ33B5EAVPPA2G2QRMGPO3LS7VCZO55UJIFFYSQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.124937699 (79h43m10.688260705s) [noise=true , address=156.146.43.225:4459 , piece_id=WBTUFC6GX6I7ABGQKIXP7VXPHIDI5TLDORBT2AFVFGV635JTCA4A, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.219375282 (79h43m10.782698289s) [upload=63815]: Next returned nil segment Jun 22 00:06:35.219398254 (79h43m10.782721278s) [upload=63815]: Waiting for error group managing segments... Jun 22 00:06:35.228097103 (79h43m10.791420115s) [noise=true , address=79.127.220.98:4402 , piece_id=TYEZTCGQRZTBHMZZDDZQTNG4DZCTFIJ3REAFDSOQWZW3UHWKSJTA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:1ko3uf7rucXvxwRA1mew5Tyyn9r1wzwffnay9Anicvq48fCWoP, address:79.127.220.98:4402): protocol: storage node overloaded, request limit: 500 Jun 22 00:06:35.273790699 (79h43m10.837113702s) [noise=true , address=77.90.31.18:4808 , piece_id=FIJJMZP53YXE7OJ33B5EAVPPA2G2QRMGPO3LS7VCZO55UJIFFYSQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:35.496191663 (79h43m11.05951466s) [noise=true , address=156.146.43.225:4459 , piece_id=WBTUFC6GX6I7ABGQKIXP7VXPHIDI5TLDORBT2AFVFGV635JTCA4A, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:35.508506927 (79h43m11.07182992s) [noise=true , address=79.127.205.234:5611 , piece_id=6NNVTKVHHZVN6BI34CVNW5DPLE7HI4F52B3TMASMKWKRM43D4QVQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:1Zck81G1h1gLrVPahfQ7VhtPdM3ciR2MsSkmttjYgaFuZDKvT5, address:79.127.205.234:5611): protocol: storage node overloaded, request limit: 2000 Jun 22 00:06:35.775728264 (79h43m11.339051266s) [noise=true , address=77.90.31.17:4803 , piece_id=MA4QUVSB5MBKWVHZSU2HTSD7HNKFCDM5LH6AOMW2JG2VJAUPNWJQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:12onacKUY32faZTFuyEjvMCZA4nVKiRKmTKh1WRFMfif2Kw1Q7Y, address:77.90.31.17:4803): protocol: storage node overloaded, request limit: 2000 Jun 22 00:06:35.915798691 (79h43m11.479121694s) [noise=true , address=104.36.82.42:6803 , piece_id=DZ5E65IJ34B66MYV7MYQ4TIJE44PWIWAKGYRCVDPYGTLKF6UKN4A, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:35.981958943 (79h43m11.545281937s) [noise=true , address=77.90.31.14:4692 , piece_id=O3GBKLZWYWSUKTZONTAFH3GPZFGTQTRQELSCMT6KB6LZFKETGKVQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:36.542263970 (79h43m12.105586963s) [noise=true , address=104.36.82.13:6196 , piece_id=IGOXQ3NJ5IDXUMUDQS7LVSSFVLZHHDL4ZDUQRCRT3SGDP6C2WBFA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:37.106276338 (79h43m12.669599339s) [noise=true , address=77.90.31.21:4887 , piece_id=OOVHZJWHOBHJUBWPZBJ7JFHAVCONGWOIQKNSWRUDRLMUVDJVW3JA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: storage node overloaded, request limit: 2000 Jun 22 00:06:38.655697328 (79h43m14.219020328s) [noise=true , address=104.36.82.5:5848 , piece_id=RESM36SSKLRSRBTHF24KRJYXRLVJ2EGYLZYMOMAERDSBHQNGZHTQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:42.966609148 (79h43m18.529932132s) [noise=true , address=104.36.82.20:6468 , piece_id=CCSQEOUXRJC7AXDSH4EASAPPSK7365ZA72OZI56C4NTVL44U46WQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:43.732966458 (79h43m19.296289462s) [noise=true , address=79.127.219.38:4036 , piece_id=SSRGW7SWOVNO4XSRIKLNDXW5F2R46FTOFQJNUAQQ34PILKHX3I4A, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:45.867417843 (79h43m21.430740847s) [noise=true , address=104.36.82.21:6493 , piece_id=GGBVGKVSAWFYTLYPNME4EBY3QAMQFA2MESSDNWAG7WLAD3NCPHKA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:50.615388344 (79h43m26.178711347s) [noise=true , address=104.36.82.53:4146 , piece_id=3R7U2WZVOFQK3YAMMBQFB62JEV4GO72YSSKZHGWUCOLGSHPYWVEQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:51.372429136 (79h43m26.935752124s) [noise=true , address=104.36.82.56:4241 , piece_id=YJESR3E5OCIDCHNDKB4EO5MPKH5IP4WIWP5FBLVF7A4SSWIDMPDQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:54.589773515 (79h43m30.153096515s) [noise=true , address=104.36.82.54:4186 , piece_id=UWTOIEDDR76OM6Y2ACCRIUTAPMMMR7MDZZJ4WKWR5DJ55IQQ67QA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:56.923953508 (79h43m32.487276509s) [noise=true , address=104.36.82.55:4204 , piece_id=TII6GSL5AQ65NVLWSTR4FV4Y3KIGOZTH6VKYSFVK3SNAQ4VAM4HA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:07:35.039734848 (79h44m10.603057859s) [noise=true , address=104.36.82.52:4108 , piece_id=CK6YCPRZFTVJNOBD6WJGXIFXJ3P5ZTKGK76CTN4HGJ3GWVYM6XPQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:12XJJZCUSHUS52YC6jhoPB4ayjsyKZT477PymPj4inkZtdUvheT, address:104.36.82.52:4108): protocol: write tcp 159.223.143.39:34012->104.36.82.52:4108: use of closed network connection; write tcp 159.223.143.39:34012->104.36.82.52:4108: use of closed network connection Jun 22 00:07:35.039830671 (79h44m10.603153681s) [seg_pos={0 0}, upload=63815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 22 00:07:35.068529987 (79h44m10.631852995s) [upload=63815]: Done uploading segments... Jun 22 00:07:35.068576498 (79h44m10.631899502s) [upload=63815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.914796574s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.........................X] duration: 1m0.001383204s FAIL (2025-06-22 00:07:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001405034s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.031383366s FAIL (2025-06-22 00:07:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 05:59.086456248 (79h42m34.649779236s) [upload=63806]: Upload finished. err: <nil> Jun 22 00:06:35.070905470 (79h43m10.634228474s) [upload=63815]: Starting upload Jun 22 00:06:35.071010688 (79h43m10.63433369s) [upload=63815]: Done starting upload Jun 22 00:06:35.071201792 (79h43m10.634524782s) [upload=63815]: Uploading segments... Jun 22 00:06:35.071643829 (79h43m10.634966841s) [upload=63815]: Got next segment. Inline: false Jun 22 00:06:35.071764824 (79h43m10.63508783s) [upload=63815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 22 00:06:35.071774624 (79h43m10.635097634s) [upload=63815]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 22 00:06:35.107204892 (79h43m10.670527901s) [seg_pos={0 0}, upload=63815]: Begin upload segment... Jun 22 00:06:35.107425462 (79h43m10.670748472s) [noise=true , address=104.36.82.42:6803 , piece_id=DZ5E65IJ34B66MYV7MYQ4TIJE44PWIWAKGYRCVDPYGTLKF6UKN4A, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.107974355 (79h43m10.671297357s) [noise=true , address=77.90.31.14:4692 , piece_id=O3GBKLZWYWSUKTZONTAFH3GPZFGTQTRQELSCMT6KB6LZFKETGKVQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.108920911 (79h43m10.672243923s) [noise=true , address=104.36.82.21:6493 , piece_id=GGBVGKVSAWFYTLYPNME4EBY3QAMQFA2MESSDNWAG7WLAD3NCPHKA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.109421565 (79h43m10.672744573s) [noise=true , address=104.36.82.20:6468 , piece_id=CCSQEOUXRJC7AXDSH4EASAPPSK7365ZA72OZI56C4NTVL44U46WQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.109844294 (79h43m10.673167296s) [noise=true , address=79.127.205.234:5611 , piece_id=6NNVTKVHHZVN6BI34CVNW5DPLE7HI4F52B3TMASMKWKRM43D4QVQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.110267970 (79h43m10.673590974s) [noise=true , address=79.127.219.38:4036 , piece_id=SSRGW7SWOVNO4XSRIKLNDXW5F2R46FTOFQJNUAQQ34PILKHX3I4A, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.111464573 (79h43m10.674787575s) [seg_pos={0 0}, upload=63815]: Done begin upload segment. Jun 22 00:06:35.111520150 (79h43m10.674843159s) [noise=true , address=77.90.31.17:4803 , piece_id=MA4QUVSB5MBKWVHZSU2HTSD7HNKFCDM5LH6AOMW2JG2VJAUPNWJQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.112035917 (79h43m10.675358922s) [noise=true , address=79.127.220.98:4402 , piece_id=TYEZTCGQRZTBHMZZDDZQTNG4DZCTFIJ3REAFDSOQWZW3UHWKSJTA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.112494883 (79h43m10.675817888s) [noise=true , address=104.36.82.55:4204 , piece_id=TII6GSL5AQ65NVLWSTR4FV4Y3KIGOZTH6VKYSFVK3SNAQ4VAM4HA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.112954016 (79h43m10.676277025s) [noise=true , address=104.36.82.13:6196 , piece_id=IGOXQ3NJ5IDXUMUDQS7LVSSFVLZHHDL4ZDUQRCRT3SGDP6C2WBFA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.113391990 (79h43m10.676714991s) [noise=true , address=104.36.82.53:4146 , piece_id=3R7U2WZVOFQK3YAMMBQFB62JEV4GO72YSSKZHGWUCOLGSHPYWVEQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.113834067 (79h43m10.677157075s) [noise=true , address=104.36.82.5:5848 , piece_id=RESM36SSKLRSRBTHF24KRJYXRLVJ2EGYLZYMOMAERDSBHQNGZHTQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.114273706 (79h43m10.677596718s) [noise=true , address=104.36.82.52:4108 , piece_id=CK6YCPRZFTVJNOBD6WJGXIFXJ3P5ZTKGK76CTN4HGJ3GWVYM6XPQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.114737587 (79h43m10.678060595s) [noise=true , address=77.90.31.21:4887 , piece_id=OOVHZJWHOBHJUBWPZBJ7JFHAVCONGWOIQKNSWRUDRLMUVDJVW3JA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.123351328 (79h43m10.68667432s) [noise=true , address=104.36.82.54:4186 , piece_id=UWTOIEDDR76OM6Y2ACCRIUTAPMMMR7MDZZJ4WKWR5DJ55IQQ67QA, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.124034043 (79h43m10.68735705s) [noise=true , address=104.36.82.56:4241 , piece_id=YJESR3E5OCIDCHNDKB4EO5MPKH5IP4WIWP5FBLVF7A4SSWIDMPDQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.124489284 (79h43m10.687812288s) [noise=true , address=77.90.31.18:4808 , piece_id=FIJJMZP53YXE7OJ33B5EAVPPA2G2QRMGPO3LS7VCZO55UJIFFYSQ, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.124937699 (79h43m10.688260705s) [noise=true , address=156.146.43.225:4459 , piece_id=WBTUFC6GX6I7ABGQKIXP7VXPHIDI5TLDORBT2AFVFGV635JTCA4A, seg_pos={0 0}, upload=63815]: Uploading piece... Jun 22 00:06:35.219375282 (79h43m10.782698289s) [upload=63815]: Next returned nil segment Jun 22 00:06:35.219398254 (79h43m10.782721278s) [upload=63815]: Waiting for error group managing segments... Jun 22 00:06:35.228097103 (79h43m10.791420115s) [noise=true , address=79.127.220.98:4402 , piece_id=TYEZTCGQRZTBHMZZDDZQTNG4DZCTFIJ3REAFDSOQWZW3UHWKSJTA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:1ko3uf7rucXvxwRA1mew5Tyyn9r1wzwffnay9Anicvq48fCWoP, address:79.127.220.98:4402): protocol: storage node overloaded, request limit: 500 Jun 22 00:06:35.273790699 (79h43m10.837113702s) [noise=true , address=77.90.31.18:4808 , piece_id=FIJJMZP53YXE7OJ33B5EAVPPA2G2QRMGPO3LS7VCZO55UJIFFYSQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:35.496191663 (79h43m11.05951466s) [noise=true , address=156.146.43.225:4459 , piece_id=WBTUFC6GX6I7ABGQKIXP7VXPHIDI5TLDORBT2AFVFGV635JTCA4A, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:35.508506927 (79h43m11.07182992s) [noise=true , address=79.127.205.234:5611 , piece_id=6NNVTKVHHZVN6BI34CVNW5DPLE7HI4F52B3TMASMKWKRM43D4QVQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:1Zck81G1h1gLrVPahfQ7VhtPdM3ciR2MsSkmttjYgaFuZDKvT5, address:79.127.205.234:5611): protocol: storage node overloaded, request limit: 2000 Jun 22 00:06:35.775728264 (79h43m11.339051266s) [noise=true , address=77.90.31.17:4803 , piece_id=MA4QUVSB5MBKWVHZSU2HTSD7HNKFCDM5LH6AOMW2JG2VJAUPNWJQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:12onacKUY32faZTFuyEjvMCZA4nVKiRKmTKh1WRFMfif2Kw1Q7Y, address:77.90.31.17:4803): protocol: storage node overloaded, request limit: 2000 Jun 22 00:06:35.915798691 (79h43m11.479121694s) [noise=true , address=104.36.82.42:6803 , piece_id=DZ5E65IJ34B66MYV7MYQ4TIJE44PWIWAKGYRCVDPYGTLKF6UKN4A, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:35.981958943 (79h43m11.545281937s) [noise=true , address=77.90.31.14:4692 , piece_id=O3GBKLZWYWSUKTZONTAFH3GPZFGTQTRQELSCMT6KB6LZFKETGKVQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:36.542263970 (79h43m12.105586963s) [noise=true , address=104.36.82.13:6196 , piece_id=IGOXQ3NJ5IDXUMUDQS7LVSSFVLZHHDL4ZDUQRCRT3SGDP6C2WBFA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:37.106276338 (79h43m12.669599339s) [noise=true , address=77.90.31.21:4887 , piece_id=OOVHZJWHOBHJUBWPZBJ7JFHAVCONGWOIQKNSWRUDRLMUVDJVW3JA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: storage node overloaded, request limit: 2000 Jun 22 00:06:38.655697328 (79h43m14.219020328s) [noise=true , address=104.36.82.5:5848 , piece_id=RESM36SSKLRSRBTHF24KRJYXRLVJ2EGYLZYMOMAERDSBHQNGZHTQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:42.966609148 (79h43m18.529932132s) [noise=true , address=104.36.82.20:6468 , piece_id=CCSQEOUXRJC7AXDSH4EASAPPSK7365ZA72OZI56C4NTVL44U46WQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:43.732966458 (79h43m19.296289462s) [noise=true , address=79.127.219.38:4036 , piece_id=SSRGW7SWOVNO4XSRIKLNDXW5F2R46FTOFQJNUAQQ34PILKHX3I4A, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:45.867417843 (79h43m21.430740847s) [noise=true , address=104.36.82.21:6493 , piece_id=GGBVGKVSAWFYTLYPNME4EBY3QAMQFA2MESSDNWAG7WLAD3NCPHKA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:50.615388344 (79h43m26.178711347s) [noise=true , address=104.36.82.53:4146 , piece_id=3R7U2WZVOFQK3YAMMBQFB62JEV4GO72YSSKZHGWUCOLGSHPYWVEQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:51.372429136 (79h43m26.935752124s) [noise=true , address=104.36.82.56:4241 , piece_id=YJESR3E5OCIDCHNDKB4EO5MPKH5IP4WIWP5FBLVF7A4SSWIDMPDQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:54.589773515 (79h43m30.153096515s) [noise=true , address=104.36.82.54:4186 , piece_id=UWTOIEDDR76OM6Y2ACCRIUTAPMMMR7MDZZJ4WKWR5DJ55IQQ67QA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:06:56.923953508 (79h43m32.487276509s) [noise=true , address=104.36.82.55:4204 , piece_id=TII6GSL5AQ65NVLWSTR4FV4Y3KIGOZTH6VKYSFVK3SNAQ4VAM4HA, seg_pos={0 0}, upload=63815]: Done uploading piece. err: <nil> Jun 22 00:07:35.039734848 (79h44m10.603057859s) [noise=true , address=104.36.82.52:4108 , piece_id=CK6YCPRZFTVJNOBD6WJGXIFXJ3P5ZTKGK76CTN4HGJ3GWVYM6XPQ, seg_pos={0 0}, upload=63815]: Done uploading piece. err: ecclient: upload failed (node:12XJJZCUSHUS52YC6jhoPB4ayjsyKZT477PymPj4inkZtdUvheT, address:104.36.82.52:4108): protocol: write tcp 159.223.143.39:34012->104.36.82.52:4108: use of closed network connection; write tcp 159.223.143.39:34012->104.36.82.52:4108: use of closed network connection Jun 22 00:07:35.039830671 (79h44m10.603153681s) [seg_pos={0 0}, upload=63815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 22 00:07:35.068529987 (79h44m10.631852995s) [upload=63815]: Done uploading segments... Jun 22 00:07:35.068576498 (79h44m10.631899502s) [upload=63815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.031411142s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 12.746723799s FAIL (2025-06-22 00:06:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.747320387s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 24.455690684s FAIL (2025-06-22 00:05:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.455730374s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 20.65356017s FAIL (2025-06-22 00:05:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.653591216s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 24.532522667s FAIL (2025-06-22 00:04:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.532694333s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 16.304839128s FAIL (2025-06-22 00:03:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.304872443s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 18.430073579s FAIL (2025-06-22 00:02:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.430097685s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X.......................................X...................] duration: 32.808240876s FAIL (2025-06-22 00:02:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 32.808263518s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X.......................................X..................] duration: 20.454871441s FAIL (2025-06-22 00:01:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.454927894s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X.......................................X.................] duration: 14.368283407s FAIL (2025-06-22 00:00:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.368328073s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X.......................................X................] duration: 16.776134184s FAIL (2025-06-22 00:00:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.776175237s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X.......................................X...............] duration: 15.921894058s FAIL (2025-06-21 23:59:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.921919052s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....X.......................................X..............] duration: 38.693810625s FAIL (2025-06-21 23:58:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 38.693838998s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......X.......................................X.............] duration: 11.109525029s FAIL (2025-06-21 23:57:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.109552358s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......X.......................................X............] duration: 14.742351692s FAIL (2025-06-21 23:57:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.742375823s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........X.......................................X...........] duration: 22.941453024s FAIL (2025-06-21 23:56:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 22.941487099s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........X.......................................X..........] duration: 13.405938927s FAIL (2025-06-21 23:55:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.405956455s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........X.......................................X.........] duration: 40.98516631s FAIL (2025-06-21 23:55:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 40.985187319s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........X.......................................X........] duration: 14.207455783s FAIL (2025-06-21 23:54:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.207481915s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............X.......................................X.......] duration: 17.747651627s FAIL (2025-06-21 23:53:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.747683742s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............X.......................................X......] duration: 12.994232423s FAIL (2025-06-21 23:52:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.994261143s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............X.......................................X.....] duration: 14.44781619s FAIL (2025-06-21 23:52:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.447835781s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............X.......................................X....] duration: 26.054641066s FAIL (2025-06-21 23:51:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 26.05466805s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................X.......................................X...] duration: 25.617668432s FAIL (2025-06-21 23:50:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.617690807s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................X.......................................X..] duration: 39.575351877s FAIL (2025-06-21 23:50:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 39.575370621s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................X.......................................X.] duration: 43.346875674s FAIL (2025-06-21 23:49:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 43.346905437s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................X.......................................X] duration: 1m0.001652031s FAIL (2025-06-21 23:48:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001671425s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 22.473363922s FAIL (2025-06-21 23:47:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 22.473390063s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 25.588304699s FAIL (2025-06-21 23:47:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.588335446s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 31.939670955s FAIL (2025-06-21 23:47:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 31.939696915s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 21.052074521s FAIL (2025-06-21 23:47:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 21.052166229s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 34.524580467s FAIL (2025-06-21 23:46:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 34.524629472s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 25.73658673s FAIL (2025-06-21 23:46:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.73660134s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 48.731443045s FAIL (2025-06-21 23:46:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 48.731469993s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 24.327549962s FAIL (2025-06-21 23:45:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 24.327573426s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 32.134279901s FAIL (2025-06-21 23:45:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 32.134309477s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 25.867817284s FAIL (2025-06-21 23:45:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.867846621s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 12.654333575s FAIL (2025-06-21 23:43:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.654355697s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 9.118581876s FAIL (2025-06-21 23:43:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.118612819s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 26.595070219s FAIL (2025-06-21 23:43:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 26.595097136s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 23.2643056s FAIL (2025-06-21 23:43:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.264343479s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 21.402172894s FAIL (2025-06-21 23:42:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 21.40220128s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 14.794509961s FAIL (2025-06-21 23:42:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.79453308s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 25.347607145s FAIL (2025-06-21 23:42:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.347634216s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 27.153958615s FAIL (2025-06-21 23:41:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.153986002s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 13.08473123s FAIL (2025-06-21 23:41:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.084753805s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 23.47281304s FAIL (2025-06-21 23:40:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.473488582s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 25.536227072s FAIL (2025-06-21 23:40:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 2994]: Done starting upload Jun 21 23:03:12.157295492 (78h39m47.720618482s) [upload=62994]: Uploading segments... Jun 21 23:03:12.157569421 (78h39m47.720892411s) [upload=62994]: Got next segment. Inline: false Jun 21 23:03:12.157636334 (78h39m47.720959333s) [upload=62994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 23:03:12.157641283 (78h39m47.720964282s) [upload=62994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 23:03:12.194202426 (78h39m47.757525429s) [seg_pos={0 0}, upload=62994]: Begin upload segment... Jun 21 23:03:12.194254676 (78h39m47.757577663s) [seg_pos={0 0}, upload=62994]: Done begin upload segment. Jun 21 23:03:12.194295158 (78h39m47.757618173s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194316742 (78h39m47.757639749s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.194749003 (78h39m47.758072012s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195040779 (78h39m47.758363779s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195268477 (78h39m47.758591484s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195482306 (78h39m47.758805305s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195569014 (78h39m47.758892014s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195908034 (78h39m47.759231036s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.195914241 (78h39m47.759237228s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196243917 (78h39m47.759566925s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196322865 (78h39m47.759645875s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196564228 (78h39m47.759887233s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196694834 (78h39m47.76001784s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.196852214 (78h39m47.760175203s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197091627 (78h39m47.760414634s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197123848 (78h39m47.760446846s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197430553 (78h39m47.760753554s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.197732318 (78h39m47.76105532s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Uploading piece... Jun 21 23:03:12.282624807 (78h39m47.845947808s) [upload=62994]: Next returned nil segment Jun 21 23:03:12.282644177 (78h39m47.845967177s) [upload=62994]: Waiting for error group managing segments... Jun 21 23:03:12.301500025 (78h39m47.864823031s) [noise=true , address=104.36.82.56:4239 , piece_id=7RJYHYTB5G4PMHJOHXLGPVRT7KWGEGW37VJZCPGEYC7DVJL22IEQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.338560176 (78h39m47.901883186s) [noise=true , address=104.36.82.54:4180 , piece_id=37IFBK5U34INHSY4WI2QEEQ47KEYA2LAEESIJ5RB2JQVT55DCEAA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:12.353567724 (78h39m47.916890712s) [noise=true , address=77.90.31.14:4666 , piece_id=WWSU4L7AKMX7SCHWUA5SGXRLJB57NDDLMTS32XIOWXW5MVRCVMGA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.425010002 (78h39m47.988333014s) [noise=true , address=45.91.116.12:5249 , piece_id=EOH6NQ5X2BF2XGSXEP32SYO5SPPRCL5GCYNVLBSKMLF3DFMCJAOQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.559667339 (78h39m48.122990329s) [noise=true , address=156.146.43.225:4467 , piece_id=25JXSBXRYBJBB7MB3WX2IHBWG7WRQ3HFKQMM57FAT5XCNKJLLSWQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:12.824470485 (78h39m48.387793479s) [noise=true , address=156.146.43.225:4454 , piece_id=6MJ66QUOQVCGK3FHZWZJ2OKI46BFI5RW6VVXMLCHE6LUAPCZJCRA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.036351387 (78h39m48.599674377s) [noise=true , address=104.36.82.9:6033 , piece_id=4FGPCQI5X22YK5RASONBYUQKFD4SGZDSNZOCLBZUPR5CNSUZZ5QA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.146623036 (78h39m48.709946021s) [noise=true , address=79.127.201.214:4320 , piece_id=NIHXDJYDWM7RF2YY75ND5CKYH2O734LUMACQGZRDM3RMR4PA6UHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.560445551 (78h39m49.123768537s) [noise=true , address=77.90.31.11:4577 , piece_id=ZTMKZBEHFTJVTFND4T367IP7KPP4TKHUYU3TAKN3DUHEWK2LSLWA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12rKdimacehmW1Tp5Zp4DJFxafLhcEeNh61cfw8Bfefr9pRG2ei, address:77.90.31.11:4577): protocol: storage node overloaded, request limit: 2000 Jun 21 23:03:13.579450750 (78h39m49.142773744s) [noise=true , address=104.36.82.5:5854 , piece_id=RAUPN2FWEC47DYZWY77JVIBS5DHGRQTRGJKEK7LCSYTLZZWPZFPQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:13.724129379 (78h39m49.287452369s) [noise=true , address=104.36.82.10:6100 , piece_id=WGS55TAVUAIIE5KTFYKG6RB2E3DMUX75463F3MPLVAHNIOQGEIHA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:14.780062664 (78h39m50.343385651s) [noise=true , address=104.36.82.16:6296 , piece_id=74WDC2WMBY62PSDBYUQ5WROBP2NNXBNEU75MLQXF5XYDROWGWQTA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:18.822633893 (78h39m54.385956894s) [noise=true , address=77.90.31.19:4845 , piece_id=GRF7FZK6GAEQKNIARYQWYYVQQGIGVNDCTI2ZG6YDRPET74V56QZQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:20.585479531 (78h39m56.148802534s) [noise=true , address=104.36.82.20:6451 , piece_id=GJAF5I74K6RXJT3ETJCJW5ZIDMDJR5VTCSBJ54V7ESM2B2H2YAVQ, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:25.391821780 (78h40m0.955144788s) [noise=true , address=104.36.82.52:4096 , piece_id=56ZOGTT7K6OA574H4KNCEKNDCRQLLOXRH2QV5C3W63SWXQ5S4C4A, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:03:39.508660492 (78h40m15.071983482s) [noise=true , address=104.36.82.53:4147 , piece_id=GCRYVKMOWUJZZIBR4XTNGZHWV4OFI3RGFKLBAOQ6YTI6W37CNFSA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: <nil> Jun 21 23:04:12.128013650 (78h40m47.691336649s) [noise=true , address=104.36.82.55:4212 , piece_id=P7ANT6LIUO45NSQYFNPIDRISCOPRUWFXS3KDJEZDQ6BUANV2QCQA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: read tcp 159.223.143.39:47784->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 23:04:12.128053405 (78h40m47.691376416s) [noise=true , address=104.36.82.12:6166 , piece_id=IPE4ZGDPLAVKLCAFBNKOCWM6DGML3WZ242BX7YG2BQYXQ5F4LDJA, seg_pos={0 0}, upload=62994]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:6166): protocol: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56704->104.36.82.12:6166: use of closed network connection Jun 21 23:04:12.128108814 (78h40m47.691431821s) [seg_pos={0 0}, upload=62994]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 23:04:12.152878890 (78h40m47.716201883s) [upload=62994]: Done uploading segments... Jun 21 23:04:12.152914585 (78h40m47.716237576s) [upload=62994]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.536259879s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)