Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 829.377935ms FAIL (2025-06-30 08:32:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 829.407448ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 650.660342ms FAIL (2025-06-30 08:31:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 650.706057ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 523.742169ms FAIL (2025-06-30 08:30:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 523.775381ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 490.521223ms FAIL (2025-06-30 08:29:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 490.547834ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 618.196892ms FAIL (2025-06-30 08:29:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 618.231907ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 550.811228ms FAIL (2025-06-30 08:28:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 550.840666ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 485.446394ms FAIL (2025-06-30 08:27:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 485.474583ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 477.193065ms FAIL (2025-06-30 08:26:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 477.23348ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 580.395657ms FAIL (2025-06-30 08:25:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 580.422076ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 630.212821ms FAIL (2025-06-30 08:24:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 630.245242ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 569.840205ms FAIL (2025-06-30 08:23:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 569.874912ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 646.310555ms FAIL (2025-06-30 08:23:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 646.41452ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 474.405562ms FAIL (2025-06-30 08:21:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 474.4314ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 647.894057ms FAIL (2025-06-30 08:20:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 647.929062ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 737.393812ms FAIL (2025-06-30 08:20:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 737.438332ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 494.169976ms FAIL (2025-06-30 08:19:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 494.195661ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 779.589253ms FAIL (2025-06-30 08:18:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 779.617005ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 561.049466ms FAIL (2025-06-30 08:17:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 561.07565ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 479.324208ms FAIL (2025-06-30 08:16:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 479.350033ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 439.094262ms FAIL (2025-06-30 08:15:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 439.123042ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 475.367784ms FAIL (2025-06-30 08:15:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 475.396782ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 530.621842ms FAIL (2025-06-30 08:14:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 530.657058ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 877.021191ms FAIL (2025-06-30 08:13:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 877.050048ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 468.68755ms FAIL (2025-06-30 08:12:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 468.716781ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 597.933005ms FAIL (2025-06-30 08:11:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 597.963631ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 590.615805ms FAIL (2025-06-30 08:10:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 590.64515ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 577.283465ms FAIL (2025-06-30 08:09:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 577.315547ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 607.970548ms FAIL (2025-06-30 08:08:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 608.000398ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 637.58544ms FAIL (2025-06-30 08:08:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 637.616106ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 496.860002ms FAIL (2025-06-30 08:06:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 496.904335ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 560.934405ms FAIL (2025-06-30 08:06:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 560.966052ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 723.767618ms FAIL (2025-06-30 08:05:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 723.79507ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 519.292772ms FAIL (2025-06-30 08:05:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 519.316186ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 641.043023ms FAIL (2025-06-30 08:05:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 641.074647ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 501.478864ms FAIL (2025-06-30 08:04:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 501.510791ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 515.363409ms FAIL (2025-06-30 08:03:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 515.388337ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 444.07368ms FAIL (2025-06-30 08:02:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 444.099629ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 538.261272ms FAIL (2025-06-30 08:02:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 538.287084ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.074927717s FAIL (2025-06-30 08:02:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectBegin Jun 30 08:02:09.521526833 (166h50m36.447821528s) [upload=133682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 30 08:02:10.707674536 (166h50m37.633969233s) [seg_pos={0 0}, upload=133682]: Begin upload segment... Jun 30 08:02:10.707741540 (166h50m37.634036235s) [seg_pos={0 0}, upload=133682]: Done begin upload segment. Jun 30 08:02:10.707765938 (166h50m37.634060615s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707787500 (166h50m37.634082202s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.707865104 (166h50m37.634159798s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708236369 (166h50m37.634531062s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708243540 (166h50m37.634538238s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708574357 (166h50m37.634869047s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708620817 (166h50m37.634915512s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708903894 (166h50m37.635198583s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.708942260 (166h50m37.635236952s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709222534 (166h50m37.635517333s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709268432 (166h50m37.635563124s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709620733 (166h50m37.635915425s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.709912660 (166h50m37.636207352s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710245117 (166h50m37.636539803s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710532064 (166h50m37.636826754s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.710854515 (166h50m37.637149206s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711081096 (166h50m37.637375791s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.711231002 (166h50m37.637525691s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Uploading piece... Jun 30 08:02:10.785256160 (166h50m37.711550846s) [upload=133682]: Next returned nil segment Jun 30 08:02:10.785270524 (166h50m37.71156521s) [upload=133682]: Waiting for error group managing segments... Jun 30 08:02:10.788413774 (166h50m37.714708454s) [noise=true , address=77.90.31.14:4685 , piece_id=HLSGQA6ZENRSMLVV5LPAIAT7K65IDOCJJ6CCLXDW5KGSNFW3MICA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.789506554 (166h50m37.715801233s) [noise=true , address=79.127.219.48:4090 , piece_id=CIIDWX2AU3NUJRN6TUF2KTUXEAFS3NIIHILIFSYBY2PBEC7QAOMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.791101128 (166h50m37.717395806s) [noise=true , address=104.36.82.59:4318 , piece_id=GQRCVHIWSFBNTUMOLJZ62IFXZSZHGT55VA4T435TC5LBNKIYHVAA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.792464349 (166h50m37.718759029s) [noise=true , address=104.36.82.42:4827 , piece_id=KZT6GEOYYMFRASYPO3IGU2GOEL3DDDFKUNYFCMGNV6VROM3W7RRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793560375 (166h50m37.719855062s) [noise=true , address=104.36.82.23:4454 , piece_id=UJFITJT72VAPTNSYQZJXZSXI2LNNWIWPD2NAWKD72A5WM72GXGGA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.793674978 (166h50m37.719969657s) [noise=true , address=104.36.82.58:4298 , piece_id=6KIEUPARRSJOU3PDHQBNZEGR33B2RG5S2G6JIA4P3XFVMSZFE64A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.805707431 (166h50m37.732002111s) [noise=true , address=77.90.31.16:4768 , piece_id=IR5XFCCZQ5FAFL7JKUQFUJUHTCPSXVITY6XA2CKPCG4FYMWXZ6WQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.808620050 (166h50m37.734914727s) [noise=true , address=104.36.82.45:4920 , piece_id=AOXJJMIIDSLQL4KGY5FLE55ALXA5PLX3XAB2ZRBORKOPKOQ7NVQQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.812691662 (166h50m37.738986335s) [noise=true , address=104.36.82.60:4366 , piece_id=3Z7JJYOWNY325WDKOL4DKIC73UTTRZEJ3I6HEFMQFJTHHBGKKXIQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.817015454 (166h50m37.743310142s) [noise=true , address=104.36.82.44:4885 , piece_id=X3Q2JD4YDEZAYUFO7WM6FUQ5OL55A54XMJUDVN4UZPR4JPYRIAUA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.826452214 (166h50m37.752746893s) [noise=true , address=104.36.82.16:4303 , piece_id=IYMPOVGPPHETH5DZ5EJYZ4I5UYKR5PND56QFNTQQUK64CFEHE43A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.830751774 (166h50m37.757046455s) [noise=true , address=104.36.82.36:4642 , piece_id=TGHALZPYKUZF33UCNMPHQSGTAWMGHSDNA3EBKXRTOOH7HF22T4VA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.837694268 (166h50m37.763988946s) [noise=true , address=104.36.82.4:4814 , piece_id=KOZ5KLCLPL37RG6NEU2KSWEGI5FMGL6QGWBPSGZW7YUCKVMUUCNA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: <nil> Jun 30 08:02:10.910774346 (166h50m37.83706904s) [noise=true , address=79.127.201.214:4301 , piece_id=CJMHXFTBFYERDEBJF6XGAYRDJFD6S4J4SW7GMUUY35PCHFQILO4A, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1dReW9bUadCFKYw28ico3Wh4v97rJmffX9ZhS3eCCRMCG9qX6h, address:79.127.201.214:4301): protocol: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47414->79.127.201.214:4301: use of closed network connection Jun 30 08:02:10.911096576 (166h50m37.837391271s) [noise=true , address=79.127.201.214:4298 , piece_id=4M74NR7LEYYSDWZMWTDRS77N543LHEEOMT7XWX7IUPVRGQD4LLMQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12sPMiBw8jUE9WA15Z5Xyt4xgiNhy1HXQqcGkUgtaLzQpFAtCdf, address:79.127.201.214:4298): protocol: expected piece hash; context deadline exceeded; EOF Jun 30 08:02:10.911315947 (166h50m37.837610638s) [noise=true , address=64.49.13.140:4511 , piece_id=5ALWHV3JDHY7HFMXUJLFBUT3ZP6SEMWQLFQ3V3ZBL2LDPEQLW4PA, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk, address:64.49.13.140:4511): protocol: read tcp 159.223.143.39:42620->64.49.13.140:4511: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911347308 (166h50m37.837642004s) [noise=true , address=64.49.13.137:4500 , piece_id=P4DZOV5FZZE2G4O67RJNOU5PJ2KAYE5YZP5V3IFF4N4DIXY2GMRQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9, address:64.49.13.137:4500): protocol: read tcp 159.223.143.39:54982->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911385931 (166h50m37.837680618s) [noise=true , address=79.127.205.233:4645 , piece_id=IETCNOUY4TXAEQRLXB32Y3ZEG3QQEAR2FVEVZSIFGWRDLICI7QJQ, seg_pos={0 0}, upload=133682]: Done uploading piece. err: ecclient: upload failed (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T, address:79.127.205.233:4645): protocol: read tcp 159.223.143.39:57052->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 30 08:02:10.911435829 (166h50m37.837730525s) [seg_pos={0 0}, upload=133682]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 30 08:02:10.983099615 (166h50m37.909394303s) [upload=133682]: Done uploading segments... Jun 30 08:02:10.983130346 (166h50m37.909425039s) [upload=133682]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration:* 1m0.074953436s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 250.883123ms FAIL (2025-06-30 07:31:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 250.91421ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 317.734048ms FAIL (2025-06-30 07:31:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 317.754611ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 256.805232ms FAIL (2025-06-30 07:30:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 256.832022ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 302.350781ms FAIL (2025-06-30 07:29:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 302.397668ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 248.943213ms FAIL (2025-06-30 07:28:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 248.988117ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 265.486428ms FAIL (2025-06-30 07:27:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 265.510751ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 275.09095ms FAIL (2025-06-30 07:27:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 275.147004ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 713.852561ms FAIL (2025-06-30 07:26:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 713.872961ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 498.513606ms FAIL (2025-06-30 07:25:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 498.546504ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 616.813374ms FAIL (2025-06-30 07:25:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 616.839255ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 854.287674ms FAIL (2025-06-30 07:24:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 854.332437ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)