Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X...........X.] duration: 23.921880725s FAIL (2025-06-21 20:11:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 23.9219058s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X...........X] duration: 1m0.028072837s FAIL (2025-06-21 20:11:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=60693]: Starting upload Jun 21 20:10:05.602866542 (75h46m41.166189557s) [upload=60693]: Done starting upload Jun 21 20:10:05.603116545 (75h46m41.166439547s) [upload=60693]: Uploading segments... Jun 21 20:10:05.603495389 (75h46m41.166818397s) [upload=60693]: Got next segment. Inline: false Jun 21 20:10:05.603566566 (75h46m41.166889558s) [upload=60693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:10:05.603573280 (75h46m41.166896286s) [upload=60693]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:10:05.639130162 (75h46m41.202453164s) [seg_pos={0 0}, upload=60693]: Begin upload segment... Jun 21 20:10:05.639231070 (75h46m41.202554057s) [seg_pos={0 0}, upload=60693]: Done begin upload segment. Jun 21 20:10:05.639292724 (75h46m41.202615728s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639338066 (75h46m41.202661071s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639416264 (75h46m41.202739267s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639873098 (75h46m41.203196115s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.639887296 (75h46m41.203210304s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640266623 (75h46m41.203589616s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640352274 (75h46m41.203675286s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640659767 (75h46m41.203982779s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.640961701 (75h46m41.20428471s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641159578 (75h46m41.204482589s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641588126 (75h46m41.204911136s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.641863297 (75h46m41.205186305s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642124427 (75h46m41.205447437s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642285917 (75h46m41.205608925s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.642656750 (75h46m41.205979758s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.643565583 (75h46m41.206888597s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.644620291 (75h46m41.207943303s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.648842639 (75h46m41.212165631s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Uploading piece... Jun 21 20:10:05.825828239 (75h46m41.389151241s) [noise=true , address=104.36.82.53:4126 , piece_id=2MKRQJEPGDJKDM7VWY5M4M6NORILAIEYYATJ7HVZEOBOMXOI2LBA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:104.36.82.53:4126): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:05.834666043 (75h46m41.397989033s) [upload=60693]: Next returned nil segment Jun 21 20:10:05.834686295 (75h46m41.398009302s) [upload=60693]: Waiting for error group managing segments... Jun 21 20:10:05.877010488 (75h46m41.440333486s) [noise=true , address=45.91.116.6:4982 , piece_id=VP45R6KXNH5U5SGUQUMAPIBY7KFDPBIR3S3VRIX27EAT6VLFBZ7Q, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:05.886336964 (75h46m41.449659967s) [noise=true , address=104.36.82.9:6039 , piece_id=I4AB6N27MYYUHWAZLJJUIBFBLOQFHDIM3YLYWPVRPNUV7VMYS2SA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:12P11WVW9mVXw5ra2gZ3Rezon6wZi8GrN5CchQN3MiVU4ts3L6V, address:104.36.82.9:6039): protocol: storage node overloaded, request limit: 2000 Jun 21 20:10:08.064514542 (75h46m43.627837553s) [noise=true , address=77.90.31.14:4670 , piece_id=7FNANMUKPM3N6YWRAOAIM4WJZJUTNF7ZBI6ZUBNEPGNGMFWOE24A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.099098391 (75h46m43.662421396s) [noise=true , address=79.127.205.245:4231 , piece_id=EJK46XUUVI5NWOAQJEW4QIJG3KRXZHB4YOQUJRCS2MIOSWKGTNLQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.489461749 (75h46m44.052784752s) [noise=true , address=104.36.82.42:6807 , piece_id=ZBQZPXUDJ4VBCOSVKWAJ6SK6UD7SLPON7Y4SSCK2DQ5STFG4XCNQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:08.650922093 (75h46m44.214245093s) [noise=true , address=77.90.31.19:4853 , piece_id=VFEON5T2KSWM2KSLTVRZTXYL42JWENZYUEUPG3LMODGZWBMVGGPQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:09.917746439 (75h46m45.481069441s) [noise=true , address=104.36.82.13:6196 , piece_id=BCHMQO5KKT5BLNWQGOHH3V43LGSE3BUQCI2SY4TTO5M5GQNOCTMQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:10.830722215 (75h46m46.394045216s) [noise=true , address=104.36.82.5:5869 , piece_id=HAEHXXZASV3DHT3QN7NTZOS6XZKHNUWBKCHWJJWSTJKXAKJKKBAA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:12.738783450 (75h46m48.302106452s) [noise=true , address=77.90.31.12:4603 , piece_id=BMUIC6KNGEDL37436GLSXW5XOO6VYOPNOSFGQCYVHSENVI2PJ7KQ, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:15.460927713 (75h46m51.024250723s) [noise=true , address=104.36.82.52:4111 , piece_id=HB7QAWN6CGNIIT4HD2GZ7JDQP5NXIJ5I2USBM66YOKTYENNOWG5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:18.393817968 (75h46m53.95714096s) [noise=true , address=104.36.82.55:4224 , piece_id=WRVNGSJV5BX4GJERLXNPAS7J7BZ5WFTM7R5H257YRCG5SP2HIYMA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:19.549174143 (75h46m55.112497151s) [noise=true , address=104.36.82.56:4253 , piece_id=5PMVGLZETMXCQ77K3I6PXACRIMBGR2CV5JGUSU5KUMYFNXVGA2GA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:22.511944593 (75h46m58.075267584s) [noise=true , address=104.36.82.37:6663 , piece_id=X7LWYIIHJFU2T2PHYR3R5TNT22UUPHEHQWZH7ZJMLS4AH4PJ6AIA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:10:48.027197558 (75h47m23.590520562s) [noise=true , address=104.36.82.14:6232 , piece_id=A2ZLUUZKRTBJRJ267QSK36C3LFZV7FPAYDDAXGIWDTNEGNV6MRRA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: <nil> Jun 21 20:11:05.567692582 (75h47m41.131015574s) [noise=true , address=104.36.82.49:4982 , piece_id=UNLV5UDNWULEIPE4QSQHHZQPK3BQKFRB5T3H6GP7ZQ5QGSNK5V6A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:104.36.82.49:4982): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567759198 (75h47m41.131082195s) [noise=true , address=104.36.82.54:4171 , piece_id=MUMABKARK4KKUKWW6XDXCQVT6O7SLFHGHE7COWC7QIYGARAA2D5A, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:11:05.567869019 (75h47m41.131192033s) [noise=true , address=104.36.82.10:6087 , piece_id=5FQG7BPZC373PFB54HYXSG2SVKOBBBSGZCLC4XDJEYQFOLQY74MA, seg_pos={0 0}, upload=60693]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:104.36.82.10:6087): protocol: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:38550->104.36.82.10:6087: use of closed network connection Jun 21 20:11:05.567932150 (75h47m41.131255161s) [seg_pos={0 0}, upload=60693]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:11:05.593626175 (75h47m41.156949181s) [upload=60693]: Done uploading segments... Jun 21 20:11:05.593657274 (75h47m41.156980265s) [upload=60693]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.028114055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 11.289480787s FAIL (2025-06-21 20:09:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 11.289510303s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 12.209297543s FAIL (2025-06-21 20:09:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 12.209324355s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 18.258572142s FAIL (2025-06-21 20:09:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.258600018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.930533763s FAIL (2025-06-21 20:08:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.930561561s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 11.474342266s FAIL (2025-06-21 20:07:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 11.474370365s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 16.279934281s FAIL (2025-06-21 20:06:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 16.279968906s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 22.820071568s FAIL (2025-06-21 20:06:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 22.820099588s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 10.674929903s FAIL (2025-06-21 20:04:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.674955996s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 17.084224818s FAIL (2025-06-21 20:04:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.084266272s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 12.859571577s FAIL (2025-06-21 20:03:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 12.859597732s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.74322897s FAIL (2025-06-21 20:02:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 16.743257654s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.031060509s FAIL (2025-06-21 20:02:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== bjectCommit Jun 21 20:00:25.864804721 (75h37m1.428127725s) [upload=60561]: Done uploading segments... Jun 21 20:00:25.864842619 (75h37m1.428165625s) [upload=60561]: Upload finished. err: <nil> Jun 21 20:01:04.928771324 (75h37m40.492094336s) [upload=60571]: Starting upload Jun 21 20:01:04.928875283 (75h37m40.492198289s) [upload=60571]: Done starting upload Jun 21 20:01:04.929112181 (75h37m40.492435195s) [upload=60571]: Uploading segments... Jun 21 20:01:04.929264872 (75h37m40.492587882s) [upload=60571]: Got next segment. Inline: false Jun 21 20:01:04.929338185 (75h37m40.492661198s) [upload=60571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 21 20:01:04.929408123 (75h37m40.492731126s) [upload=60571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 20:01:04.971255191 (75h37m40.534578198s) [seg_pos={0 0}, upload=60571]: Begin upload segment... Jun 21 20:01:04.971349357 (75h37m40.53467236s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.971871707 (75h37m40.535194719s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972277631 (75h37m40.535600636s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972619095 (75h37m40.5359421s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.972994779 (75h37m40.536317783s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973334591 (75h37m40.536657604s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.973799745 (75h37m40.537122758s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.974168284 (75h37m40.53749129s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975169359 (75h37m40.538492393s) [seg_pos={0 0}, upload=60571]: Done begin upload segment. Jun 21 20:01:04.975221756 (75h37m40.538544766s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.975667597 (75h37m40.538990608s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976066241 (75h37m40.539389254s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976434219 (75h37m40.539757226s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.976792070 (75h37m40.540115076s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977138766 (75h37m40.540461775s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977617146 (75h37m40.540940157s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.977958582 (75h37m40.54128159s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978298587 (75h37m40.541621595s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:04.978698247 (75h37m40.542021255s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Uploading piece... Jun 21 20:01:05.082546722 (75h37m40.64586973s) [upload=60571]: Next returned nil segment Jun 21 20:01:05.082569073 (75h37m40.645892084s) [upload=60571]: Waiting for error group managing segments... Jun 21 20:01:05.307002269 (75h37m40.870325276s) [noise=true , address=79.127.205.246:4282 , piece_id=RAMAW54XVC445QN2R7O7MJQ6ZWFVIJTALVWN3MLQN3TH4WVLVWKQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1h57yhefx9MHNAFXV3CZAzTopF5v61K6SyEdXh9HqjxndPw1mm, address:79.127.205.246:4282): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.328768725 (75h37m40.892091734s) [noise=true , address=45.91.116.6:4982 , piece_id=S4NRHOZOYPFB6ANK74PNKI5O2DJTVUKZKWWIG33UOZGMW64LNW5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:125rPFZnyTxsa7RBQfAkXtkVCdUJUykq644EK4Ny6HQ1Q7NzrF3, address:45.91.116.6:4982): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:05.782721883 (75h37m41.346044891s) [noise=true , address=77.90.31.14:4683 , piece_id=GGMR2TUXRTFD2AHMMJ73XFOYGEQ6H5PI66GG3WKIXLGM3AKRYGOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.523999317 (75h37m42.087322325s) [noise=true , address=79.127.205.245:4225 , piece_id=BWKYCDIHHNEVMQDKTAEI7MD3F7GOC4VD7N5DPTI3SKAMUOVWDOWA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:06.585239448 (75h37m42.148562448s) [noise=true , address=104.36.82.11:6127 , piece_id=MJ5ZAZNQEDCHLF6CKTAOUTOL5ZFVKO64MNWL5FOUDRXMUBDH7LMQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.183130402 (75h37m42.746453407s) [noise=true , address=104.36.82.13:6210 , piece_id=OHQKKJQ4S7EXRLHDBTQQWICA7DQNWJIEUSLAZPOYRNMEIX5QFBFQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:07.724069275 (75h37m43.287392264s) [noise=true , address=104.36.82.6:5898 , piece_id=KZYE2YG4GSHFY6FO4GPFSGRA4RKIZDWYBDSNVAS4R2UCDNBWUE5A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:08.035349036 (75h37m43.598672038s) [noise=true , address=104.36.82.56:4250 , piece_id=Z5DAB4WO3EL26CUM5GR4X2D2PQCX4BN4WE6WUEIR5WOXFKQQUYCQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD, address:104.36.82.56:4250): protocol: storage node overloaded, request limit: 2000 Jun 21 20:01:08.291414275 (75h37m43.854737344s) [noise=true , address=104.36.82.9:6020 , piece_id=WSIT5JQ73KRSXOM4KHZCV3APYWRB5CWOXNBSR66ANXJBS6QEUIOA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:10.853049642 (75h37m46.41637265s) [noise=true , address=104.36.82.15:6272 , piece_id=YS3JWEHCNQWYOLMPFGPC76B662C5QWUF7LZWP75SNKTE5GEZONFA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:11.841260265 (75h37m47.404583267s) [noise=true , address=79.127.219.38:4029 , piece_id=6NOCROU7AGBVG5KB5AUM566RQEADM7CFWWUK4ZZFAWR2MFXGTZTQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:16.176493236 (75h37m51.739816237s) [noise=true , address=104.36.82.53:4133 , piece_id=4JOF3SCPHD2IEOTSW7UF6NJZ5FI4M5PVF46YMLZ45ZYNJYNMJQYQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:18.114782582 (75h37m53.678105572s) [noise=true , address=104.36.82.22:6514 , piece_id=NGQA4S5D6OF3B4TEQD7NIIDY334HDXLW3XJ2QKNUKCY5ARI3SSQA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:21.995601374 (75h37m57.558924376s) [noise=true , address=104.36.82.55:4220 , piece_id=QYEUHTISVBSTEOR4NMRRWXU62QTX72T7J7MNXHYUPHX4I4KYR37A, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:33.842824433 (75h38m9.406147419s) [noise=true , address=104.36.82.10:6080 , piece_id=XBWRYSBMIHQTJS3SIPGCPDJTRVUCQYWUMTK7VNDNFJ66ZA3A6TSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:01:34.970318938 (75h38m10.533641926s) [noise=true , address=104.36.82.54:4168 , piece_id=YSPFXHSK3NM3Q4KQS6UCSWX2HZVSEVHWABRZHOYYVBXI2NBWXDXQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: <nil> Jun 21 20:02:04.889847409 (75h38m40.453170398s) [noise=true , address=104.36.82.21:6483 , piece_id=VYYYQEF7P6DISBU7366Q6GFCVZRQMXR5FYIAYGNNW7PR7GUHV5CA, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:12gxKuBALDLDVDfRGBvMZymwGyPfPtV1BsHdBPqZs34ugvvYacG, address:104.36.82.21:6483): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889902236 (75h38m40.453225229s) [noise=true , address=104.36.82.42:6810 , piece_id=ASJPGPX5NC2XTMYD4J4NWKDAA3OLILSBV4LU4BGQ33K6JAQXMZSQ, seg_pos={0 0}, upload=60571]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:6810): protocol: expected piece hash; context deadline exceeded; EOF Jun 21 20:02:04.889946644 (75h38m40.453269714s) [seg_pos={0 0}, upload=60571]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jun 21 20:02:04.918413368 (75h38m40.481736373s) [upload=60571]: Done uploading segments... Jun 21 20:02:04.918447959 (75h38m40.481770959s) [upload=60571]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.031091112s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 161.91491ms FAIL (2025-06-21 16:39:56 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 161.944468ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 248.354984ms FAIL (2025-06-21 16:38:53 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 248.380108ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 196.261381ms FAIL (2025-06-21 16:37:52 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 196.282092ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 141.603682ms FAIL (2025-06-21 16:37:03 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 141.634606ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 146.696744ms FAIL (2025-06-21 16:36:46 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 146.738158ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 234.015278ms FAIL (2025-06-21 16:36:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 234.162993ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 163.284001ms FAIL (2025-06-21 16:35:56 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 163.324956ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 159.830856ms FAIL (2025-06-21 16:35:21 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 159.858308ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 307.645332ms FAIL (2025-06-21 16:34:49 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 307.67651ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 148.501443ms FAIL (2025-06-21 16:33:51 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 148.529663ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 196.616268ms FAIL (2025-06-21 16:33:32 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 196.644965ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 217.67486ms FAIL (2025-06-21 16:32:35 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 217.694474ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 191.89612ms FAIL (2025-06-21 16:31:47 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 191.922807ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 211.146102ms FAIL (2025-06-21 16:31:29 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 211.174815ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 177.583151ms FAIL (2025-06-21 16:31:07 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 177.609635ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 151.416836ms FAIL (2025-06-21 16:30:51 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 151.464368ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 382.933394ms FAIL (2025-06-21 16:30:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 382.961205ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 152.589777ms FAIL (2025-06-21 16:29:58 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 152.62008ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 195.334691ms FAIL (2025-06-21 16:29:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 195.363243ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 168.691256ms FAIL (2025-06-21 16:28:47 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 168.714704ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 146.967237ms FAIL (2025-06-21 16:27:53 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 146.992026ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 149.593534ms FAIL (2025-06-21 16:27:03 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 149.624464ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 438.72092ms FAIL (2025-06-21 16:26:21 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 438.744819ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 197.109436ms FAIL (2025-06-21 16:25:33 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 197.143509ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 195.866819ms FAIL (2025-06-21 16:25:08 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 195.894705ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 227.483688ms FAIL (2025-06-21 16:24:46 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 227.505085ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 162.494318ms FAIL (2025-06-21 16:23:58 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 162.551362ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 144.839107ms FAIL (2025-06-21 16:23:13 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 144.86222ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 194.163581ms FAIL (2025-06-21 16:22:14 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 194.207579ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 148.741869ms FAIL (2025-06-21 16:21:49 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 148.769311ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 230.355806ms FAIL (2025-06-21 16:21:04 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 230.405645ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 229.387675ms FAIL (2025-06-21 16:20:16 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 229.413839ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 181.170215ms FAIL (2025-06-21 16:19:48 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 181.199343ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 171.26538ms FAIL (2025-06-21 16:19:23 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 171.287329ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 167.330729ms FAIL (2025-06-21 16:18:26 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 167.354454ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 163.717692ms FAIL (2025-06-21 16:17:56 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 163.767329ms edge authservice http bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)