Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.332948067s FAIL (2025-07-06 23:36:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.332968698s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.316122107s FAIL (2025-07-06 23:36:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.316158272s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 466.218574ms FAIL (2025-07-06 23:35:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 466.249107ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.149994163s FAIL (2025-07-06 23:34:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.150037346s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.304364719s FAIL (2025-07-06 23:34:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.304393772s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X................................................X..........] duration: 1.020469372s FAIL (2025-07-06 23:33:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.020496367s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X................................................X.........] duration: 1.400807013s FAIL (2025-07-06 23:32:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.400828492s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X................................................X........] duration: 1.091607492s FAIL (2025-07-06 23:31:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.091630332s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X................................................X.......] duration: 972.941052ms FAIL (2025-07-06 23:31:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 972.976064ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X................................................X......] duration: 999.993263ms FAIL (2025-07-06 23:30:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.000048875s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X................................................X.....] duration: 879.887963ms FAIL (2025-07-06 23:29:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 879.90953ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X................................................X....] duration: 502.478899ms FAIL (2025-07-06 23:28:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 502.497621ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X................................................X...] duration: 947.806261ms FAIL (2025-07-06 23:27:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 947.830222ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X................................................X..] duration: 537.420182ms FAIL (2025-07-06 23:27:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 537.448145ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X................................................X.] duration: 1.512213963s FAIL (2025-07-06 23:26:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.512255306s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X................................................X] duration: 30.000826255s FAIL (2025-07-06 23:26:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000847918s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.960118782s FAIL (2025-07-06 23:25:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.960149302s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.523289448s FAIL (2025-07-06 23:24:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.523304722s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 758.048456ms FAIL (2025-07-06 23:23:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 758.074155ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 489.662554ms FAIL (2025-07-06 23:23:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 489.67653ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 454.160502ms FAIL (2025-07-06 23:21:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.184607ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 563.276015ms FAIL (2025-07-06 23:21:31 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 563.300176ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.207821765s FAIL (2025-07-06 23:21:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.207838685s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 481.049978ms FAIL (2025-07-06 23:21:05 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 481.080577ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 716.708195ms FAIL (2025-07-06 23:20:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 716.754804ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 772.59057ms FAIL (2025-07-06 23:20:02 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 772.679705ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 491.573403ms FAIL (2025-07-06 23:19:38 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 491.610702ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.200253991s FAIL (2025-07-06 23:19:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.200279505s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 643.208804ms FAIL (2025-07-06 23:19:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 643.232014ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 490.776462ms FAIL (2025-07-06 23:18:39 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 490.805998ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 564.586309ms FAIL (2025-07-06 23:18:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 564.623418ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 558.419663ms FAIL (2025-07-06 23:18:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 558.440098ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.783067957s FAIL (2025-07-06 23:17:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.783093346s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 447.165916ms FAIL (2025-07-06 23:17:41 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 447.196787ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 511.472573ms FAIL (2025-07-06 23:16:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 511.52315ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 453.653616ms FAIL (2025-07-06 23:16:46 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 453.679011ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 3.126888733s FAIL (2025-07-06 23:16:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.12694138s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 451.814058ms FAIL (2025-07-06 23:15:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 451.863178ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 410.425197ms FAIL (2025-07-06 23:14:59 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 410.45623ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 846.780663ms FAIL (2025-07-06 23:14:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 846.808527ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 856.523996ms FAIL (2025-07-06 23:14:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 856.565777ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 421.685048ms FAIL (2025-07-06 23:13:58 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 421.71303ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 992.753519ms FAIL (2025-07-06 23:13:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 992.775568ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 426.91874ms FAIL (2025-07-06 23:13:08 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 426.946831ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 512.716965ms FAIL (2025-07-06 23:12:49 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 512.743867ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 3.800060831s FAIL (2025-07-06 23:11:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.80008494s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 497.770706ms FAIL (2025-07-06 23:11:51 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 497.796762ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 426.825776ms FAIL (2025-07-06 23:11:32 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 426.856374ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.002849484s FAIL (2025-07-06 23:11:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.00288272s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 480.58148ms FAIL (2025-07-06 23:10:50 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 25h25m55.361742804s) [upload=260769]: Got next segment. Inline: false Jul 6 22:37:28.435492991 (325h25m55.361787688s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 6 22:37:28.435500377 (325h25m55.361795073s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 6 22:37:28.475468435 (325h25m55.401763124s) [seg_pos={0 0}, upload=260769]: Begin upload segment... Jul 6 22:37:28.475525431 (325h25m55.401820125s) [seg_pos={0 0}, upload=260769]: Done begin upload segment. Jul 6 22:37:28.475557145 (325h25m55.401851842s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475571255 (325h25m55.40186595s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.475685830 (325h25m55.401980528s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476158395 (325h25m55.402453091s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476170039 (325h25m55.402464735s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476597058 (325h25m55.402891761s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476682225 (325h25m55.402976925s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.476961731 (325h25m55.403256427s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477131527 (325h25m55.403426215s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477303263 (325h25m55.403597966s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477550266 (325h25m55.403844955s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477659395 (325h25m55.403954094s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.477995661 (325h25m55.404290357s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478011022 (325h25m55.404305719s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478370683 (325h25m55.404665381s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478440835 (325h25m55.404735525s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.478862247 (325h25m55.40515694s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.479317662 (325h25m55.405612372s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Uploading piece... Jul 6 22:37:28.583530796 (325h25m55.509825491s) [upload=260769]: Next returned nil segment Jul 6 22:37:28.583554476 (325h25m55.509849183s) [upload=260769]: Waiting for error group managing segments... Jul 6 22:37:28.593188159 (325h25m55.51948285s) [noise=true , address=77.90.31.13:4656 , piece_id=JZDKOUXFVBIPWDNJJ2CEE35WTNBQNJNQKEOOK2JPEC2XXNSGNSSQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594237326 (325h25m55.520532011s) [noise=true , address=77.90.31.13:4640 , piece_id=Y3EIFIJ66A55KSQRC7S3DQMZQFJWIDX3AJAV42PRKEDB7NIKKMHQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.594498446 (325h25m55.520793119s) [noise=true , address=77.90.31.14:4674 , piece_id=6RZYVKTXPWI4SPC7SNIZHKLOWM54MSW2BFKYLF2SA3SEPDU6KESQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.595893536 (325h25m55.522188216s) [noise=true , address=77.90.31.15:4702 , piece_id=XX37CWXF2TWIQENXTUNVD32GZKJCQEEVULN62E6Y7Q7NWWZST2EQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.596234256 (325h25m55.52252893s) [noise=true , address=77.90.31.12:4612 , piece_id=R3TVFBEB7F5CCEEMCN4HU3TIMHDXOZNQIOA7IQDGQTAVELLX2P4A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597106515 (325h25m55.523401192s) [noise=true , address=104.36.82.60:4346 , piece_id=7TEOESUASIA2L5R2CFK5K4D3MWVS4VUXIEDP7YYSD5UARAVQE3UA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.597199390 (325h25m55.523494076s) [noise=true , address=104.36.82.34:4565 , piece_id=2NP5LIKNIK7JZCYNDG7HB4SFVV2A76Q2RIYWBZJNEXVL5QDUVSQA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.600500895 (325h25m55.526795583s) [noise=true , address=45.91.116.9:4273 , piece_id=YVPP2NYDGDMT2GMAZ47CVEX7RLGFAGTUNQGMDXBVQG4H6JS7KYKA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.603000635 (325h25m55.529295319s) [noise=true , address=104.36.82.3:4793 , piece_id=XD57IIBTCUYFTKWY2AJ3FUZK7XZ57FFHRBEDXJWGY6LOLRWXU57A, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.605156008 (325h25m55.531450688s) [noise=true , address=104.36.82.10:4096 , piece_id=KKRSEJXNXXWTHRPZRECRUKGJFZS6UKW2HF2JRRZFHE6CJVARUSAQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.609885234 (325h25m55.53617992s) [noise=true , address=104.36.82.12:4179 , piece_id=XQ4VZL2MNVUVWLI6IBCQIFXEUYQLQAVYXZV3LWURQZGIXXBRMRPA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.610607908 (325h25m55.536902595s) [noise=true , address=77.90.31.21:4894 , piece_id=2OJ7LF7F6Q33MLU3IC5YVX2WNSTJS6SP46WAGJTB6TNUFIPX62YA, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.612902086 (325h25m55.539196759s) [noise=true , address=45.91.116.6:4004 , piece_id=TX3XSBP5QPMBCNOA4SDYAMJDBUIWNPHJFTNK4KHMMM64Y2LL2PQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622244006 (325h25m55.548538693s) [noise=true , address=104.36.82.36:4638 , piece_id=ZWY7EYQCRPSKMNFV7DHETGHHDBO25W7KPQFGIO22U6X46VFOOTPQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: <nil> Jul 6 22:37:28.622275216 (325h25m55.548569903s) [seg_pos={0 0}, upload=260769]: Segment reached optimal threshold of 14 pieces. Jul 6 22:37:28.622517876 (325h25m55.548812564s) [noise=true , address=104.36.82.8:4976 , piece_id=RNXKITMHZWIEHQ2CROSPEX35A6KM4T5FK6AIXPQNOR477QYKEBTQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622703448 (325h25m55.548998135s) [noise=true , address=79.127.205.234:4598 , piece_id=XYEX6GB7FKAEK3ZLHCVLANXC6KDVGSRTXP6VFKA5R2JT6Z56SPCQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1em3M1WGGYB3mGai2ayAnwWFQU3c6LA3MJQf3PekZcvssk5qaT): protocol: read tcp 159.223.143.39:51948->79.127.205.234:4598: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622743290 (325h25m55.54903797s) [noise=true , address=64.49.13.169:4710 , piece_id=YJYX6BUYUN5WJT2ASVDGD3IIEPFG5MDCE2N5GD77R4N4LP3PFNQQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: read tcp 159.223.143.39:47164->64.49.13.169:4710: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 6 22:37:28.622759239 (325h25m55.549053917s) [noise=true , address=104.36.82.5:4868 , piece_id=CUF55MM7EEREBC47WUUH4RBUYOSFNQ56NOYJVMN3AASOF4JXP2UQ, seg_pos={0 0}, upload=260769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Jul 6 22:37:28.622771253 (325h25m55.54906594s) [seg_pos={0 0}, upload=260769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 6 22:37:28.622851917 (325h25m55.549146608s) [upload=260769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 6 22:37:28.622857048 (325h25m55.549151734s) [upload=260769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 6 22:37:28.674888622 (325h25m55.601183318s) [upload=260769]: Done uploading segments... Jul 6 22:37:28.674913445 (325h25m55.601208134s) [upload=260769]: Upload finished. err: <nil> Duration: 480.611786ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)