Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................X...........X..........................] duration: 24.291828123s FAIL (2025-06-21 20:33:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 24.291963321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................X...........X.........................] duration: 32.785095055s FAIL (2025-06-21 20:32:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 32.785126815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................X...........X........................] duration: 28.600434712s FAIL (2025-06-21 20:31:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 28.600465624s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................X...........X.......................] duration: 9.835063044s FAIL (2025-06-21 20:30:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.835094645s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................X...........X......................] duration: 18.154400925s FAIL (2025-06-21 20:29:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.154433043s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................X...........X.....................] duration: 23.102523773s FAIL (2025-06-21 20:28:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 23.10255471s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................X...........X....................] duration: 27.092209567s FAIL (2025-06-21 20:27:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.092240928s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................X...........X...................] duration: 24.287283946s FAIL (2025-06-21 20:26:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 24.287313277s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................X...........X..................] duration: 19.412019074s FAIL (2025-06-21 20:25:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.412066965s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................X...........X.................] duration: 15.377422554s FAIL (2025-06-21 20:24:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.377446313s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................X...........X................] duration: 18.882619453s FAIL (2025-06-21 20:23:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.882692674s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................X...........X...............] duration: 22.256548969s FAIL (2025-06-21 20:22:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 22.256578286s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................X...........X..............] duration: 19.789731478s FAIL (2025-06-21 20:21:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.789757582s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................X...........X.............] duration: 25.879294238s FAIL (2025-06-21 20:20:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.879321862s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................X...........X............] duration: 9.85608251s FAIL (2025-06-21 20:19:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.856105724s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................X...........X...........] duration: 28.050748236s FAIL (2025-06-21 20:19:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 28.050802946s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X...........X..........] duration: 15.58526699s FAIL (2025-06-21 20:18:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.585297471s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................X...........X.........] duration: 19.678500308s FAIL (2025-06-21 20:17:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.678529138s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X...........X........] duration: 15.830333344s FAIL (2025-06-21 20:16:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.83036763s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................X...........X.......] duration: 18.899167806s FAIL (2025-06-21 20:15:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.899192573s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X...........X......] duration: 29.201249838s FAIL (2025-06-21 20:14:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 29.201276461s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X...........X.....] duration: 9.443636861s FAIL (2025-06-21 20:14:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.443668744s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X...........X....] duration: 14.064363297s FAIL (2025-06-21 20:13:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.064396921s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................X...........X...] duration: 17.726500126s FAIL (2025-06-21 20:12:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.72661827s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X...........X..] duration: 14.719725463s FAIL (2025-06-21 20:11:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.719753612s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X...........X.] duration: 23.921880725s FAIL (2025-06-21 20:11:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 23.9219058s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X...........X] duration: 1m0.028072837s FAIL (2025-06-21 20:11:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.028114055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 11.289480787s FAIL (2025-06-21 20:09:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 11.289510303s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 12.209297543s FAIL (2025-06-21 20:09:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 12.209324355s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 18.258572142s FAIL (2025-06-21 20:09:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.258600018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.930533763s FAIL (2025-06-21 20:08:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.930561561s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 11.474342266s FAIL (2025-06-21 20:07:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 11.474370365s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 16.279934281s FAIL (2025-06-21 20:06:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 16.279968906s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 22.820071568s FAIL (2025-06-21 20:06:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 22.820099588s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 10.674929903s FAIL (2025-06-21 20:04:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.674955996s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 17.084224818s FAIL (2025-06-21 20:04:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.084266272s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 12.859571577s FAIL (2025-06-21 20:03:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 12.859597732s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.74322897s FAIL (2025-06-21 20:02:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 16.743257654s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.031060509s FAIL (2025-06-21 20:02:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.031091112s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 161.91491ms FAIL (2025-06-21 16:39:56 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 161.944468ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 248.354984ms FAIL (2025-06-21 16:38:53 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 248.380108ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 196.261381ms FAIL (2025-06-21 16:37:52 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 196.282092ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 141.603682ms FAIL (2025-06-21 16:37:03 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 141.634606ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 146.696744ms FAIL (2025-06-21 16:36:46 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 146.738158ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 234.015278ms FAIL (2025-06-21 16:36:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 234.162993ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 163.284001ms FAIL (2025-06-21 16:35:56 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 163.324956ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 159.830856ms FAIL (2025-06-21 16:35:21 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 159.858308ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 307.645332ms FAIL (2025-06-21 16:34:49 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 307.67651ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 148.501443ms FAIL (2025-06-21 16:33:51 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 148.529663ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 196.616268ms FAIL (2025-06-21 16:33:32 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 196.644965ms edge authservice http bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)