Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 508.787584ms FAIL (2025-07-07 22:57:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 508.817317ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.402830609s FAIL (2025-07-07 22:56:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.402851653s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 725.373394ms FAIL (2025-07-07 22:55:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 725.39696ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 477.014026ms FAIL (2025-07-07 22:55:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 477.038967ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 532.804778ms FAIL (2025-07-07 22:54:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 532.830823ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.26179704s FAIL (2025-07-07 22:53:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.261818628s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 494.342568ms FAIL (2025-07-07 22:52:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 494.395051ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 552.179986ms FAIL (2025-07-07 22:52:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 552.207472ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 563.647807ms FAIL (2025-07-07 22:51:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 563.741724ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.329828797s FAIL (2025-07-07 22:50:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.329920368s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 543.759545ms FAIL (2025-07-07 22:50:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 543.792408ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 470.225796ms FAIL (2025-07-07 22:49:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 470.250254ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.360726495s FAIL (2025-07-07 22:48:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.360753262s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 886.694886ms FAIL (2025-07-07 22:47:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 886.710181ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 483.91895ms FAIL (2025-07-07 22:46:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 483.940975ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 631.582111ms FAIL (2025-07-07 22:46:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 631.605383ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.226669769s FAIL (2025-07-07 22:45:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.226694146s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 825.126764ms FAIL (2025-07-07 22:45:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 825.149941ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 452.870384ms FAIL (2025-07-07 22:44:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 452.904393ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.282518462s FAIL (2025-07-07 22:43:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.282546211s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 696.336253ms FAIL (2025-07-07 22:43:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 697.428419ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 683.40316ms FAIL (2025-07-07 22:42:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 683.45127ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.193436416s FAIL (2025-07-07 22:41:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.193452661s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001387687s FAIL (2025-07-07 22:41:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001409405s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 536.226954ms FAIL (2025-07-07 21:20:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 536.253287ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 509.465233ms FAIL (2025-07-07 21:19:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 509.489466ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 396.855139ms FAIL (2025-07-07 21:19:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 396.886251ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 268.91199ms FAIL (2025-07-07 21:18:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 268.938488ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 452.636899ms FAIL (2025-07-07 21:18:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 452.684736ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 288.756558ms FAIL (2025-07-07 21:17:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 288.779827ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 261.632493ms FAIL (2025-07-07 21:17:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 261.661337ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 574.052075ms FAIL (2025-07-07 21:17:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 574.146849ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 271.048254ms FAIL (2025-07-07 21:16:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 271.06937ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 231.92227ms FAIL (2025-07-07 21:16:14 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 231.949854ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 582.920514ms FAIL (2025-07-07 21:15:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 582.942562ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 547.173776ms FAIL (2025-07-07 21:15:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 547.212436ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 275.956179ms FAIL (2025-07-07 21:15:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 275.986762ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 502.607292ms FAIL (2025-07-07 21:14:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 503.132787ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 256.329074ms FAIL (2025-07-07 21:14:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 256.357643ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 480.621765ms FAIL (2025-07-07 21:13:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 480.646591ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 238.050461ms FAIL (2025-07-07 21:13:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 238.083946ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 570.87189ms FAIL (2025-07-07 21:13:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 570.896364ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 231.258197ms FAIL (2025-07-07 21:12:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 231.293995ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 523.077378ms FAIL (2025-07-07 21:12:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 523.587082ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 247.302728ms FAIL (2025-07-07 21:12:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 247.352478ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 253.845028ms FAIL (2025-07-07 21:11:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 253.873503ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 494.763207ms FAIL (2025-07-07 21:11:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 494.787816ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 596.295976ms FAIL (2025-07-07 21:10:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 596.324616ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 245.514465ms FAIL (2025-07-07 21:10:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 245.543605ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 233.936679ms FAIL (2025-07-07 21:09:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 233.96907ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)