Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [........X.........XXX........X..X....X.X.X......X......X....] duration: 677.944682ms FAIL (2025-07-09 01:40:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 677.970958ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...XX.XXX.] duration: 51.74683453s FAIL (2025-07-09 01:39:42 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 51.746850186s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.........X.........XXX........X..X....X.X.X......X......X...] duration: 13.01896238s FAIL (2025-07-09 01:39:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.018990775s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 23 out of 60 is failed (threshold=3) [X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...XX.XXX] duration: 1m0.000482305s FAIL (2025-07-09 01:38:50 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000511044s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........X.........XXX........X..X....X.X.X......X......X..] duration: 623.691975ms FAIL (2025-07-09 01:38:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 623.704608ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........X.........XXX........X..X....X.X.X......X......X.] duration: 29.291512055s FAIL (2025-07-09 01:38:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 29.291536316s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [.X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...XX.XX] duration: 1m0.000693141s FAIL (2025-07-09 01:37:49 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000751612s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............X.........XXX........X..X....X.X.X......X......X] duration: 30.000801836s FAIL (2025-07-09 01:36:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000831417s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [..X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...XX.X] duration: 1m0.000601565s FAIL (2025-07-09 01:36:49 UTC) context deadline exceeded Duration: 1m0.000623897s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X............X.........XXX........X..X....X.X.X......X......] duration: 492.479101ms FAIL (2025-07-09 01:36:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 492.533738ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X............X.........XXX........X..X....X.X.X......X.....] duration: 529.955669ms FAIL (2025-07-09 01:35:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 530.005491ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...XX.] duration: 59.632954621s FAIL (2025-07-09 01:35:49 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 59.632969841s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X............X.........XXX........X..X....X.X.X......X....] duration: 12.734112449s FAIL (2025-07-09 01:35:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.734133466s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [X...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...XX] duration: 1m0.000807344s FAIL (2025-07-09 01:34:49 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000821778s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X............X.........XXX........X..X....X.X.X......X...] duration: 637.575902ms FAIL (2025-07-09 01:34:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 637.60172ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....X............X.........XXX........X..X....X.X.X......X..] duration: 16.556862324s FAIL (2025-07-09 01:34:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.557006914s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....X............X.........XXX........X..X....X.X.X......X.] duration: 28.913198075s FAIL (2025-07-09 01:33:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 28.913223257s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [XX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...X] duration: 1m0.001457102s FAIL (2025-07-09 01:33:49 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001490562s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......X............X.........XXX........X..X....X.X.X......X] duration: 30.000599243s FAIL (2025-07-09 01:33:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000623127s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX...] duration: 46.60956951s FAIL (2025-07-09 01:32:49 UTC) operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 46.609620599s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX..] duration: 48.27537109s FAIL (2025-07-09 01:32:03 UTC) operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 48.275385667s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X............X.........XXX........X..X....X.X.X......] duration: 567.792436ms FAIL (2025-07-09 01:32:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 567.81739ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X............X.........XXX........X..X....X.X.X.....] duration: 28.022232354s FAIL (2025-07-09 01:31:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 28.022255071s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX.] duration: 53.394222193s FAIL (2025-07-09 01:31:14 UTC) operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 53.394251254s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........X............X.........XXX........X..X....X.X.X....] duration: 8.505536684s FAIL (2025-07-09 01:31:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.505570999s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [.X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..XX] duration: 1m0.00177558s FAIL (2025-07-09 01:30:21 UTC) operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001791858s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X.........X............X.........XXX........X..X....X.X.X...] duration: 1.101995579s FAIL (2025-07-09 01:30:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.102092274s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X.........X............X.........XXX........X..X....X.X.X..] duration: 10.310695269s FAIL (2025-07-09 01:29:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.310722964s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [..X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..X] duration: 1m0.00077836s FAIL (2025-07-09 01:29:21 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000801427s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X.........X............X.........XXX........X..X....X.X.X.] duration: 6.244688718s FAIL (2025-07-09 01:28:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.244724539s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X.........X............X.........XXX........X..X....X.X.X] duration: 30.000553204s FAIL (2025-07-09 01:28:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000628242s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [...X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X..] duration: 46.052027729s FAIL (2025-07-09 01:28:07 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 46.05209487s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X...X.........X............X.........XXX........X..X....X.X.] duration: 8.265265154s FAIL (2025-07-09 01:27:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.265283873s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X...X.........X............X.........XXX........X..X....X.X] duration: 30.001141012s FAIL (2025-07-09 01:27:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001164927s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [X...X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X.] duration: 55.743858263s FAIL (2025-07-09 01:27:21 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 55.743887699s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X...X.........X............X.........XXX........X..X....X.] duration: 10.141998091s FAIL (2025-07-09 01:26:54 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.142236579s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 403.26007ms FAIL (2025-07-09 01:26:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== errs: <nil> Jul 9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments... Jul 9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil> Jul 9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload Jul 9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload Jul 9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments... Jul 9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false Jul 9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment... Jul 9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment. Jul 9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment Jul 9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments... Jul 9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection Jul 9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection Jul 9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection Jul 9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments... Jul 9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 403.308368ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X...X.........X............X.........XXX........X..X....X] duration: 30.001681349s FAIL (2025-07-09 01:26:12 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00170749s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [XX...X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.X] duration: 1m0.000312795s FAIL (2025-07-09 01:26:10 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000327316s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 568.023564ms FAIL (2025-07-09 01:25:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== errs: <nil> Jul 9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments... Jul 9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil> Jul 9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload Jul 9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload Jul 9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments... Jul 9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false Jul 9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment... Jul 9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment. Jul 9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment Jul 9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments... Jul 9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection Jul 9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection Jul 9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection Jul 9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments... Jul 9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 568.054658ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [.XX...X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX.] duration: 46.320098489s FAIL (2025-07-09 01:25:10 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 46.320116792s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [....X...X.........X............X.........XXX........X..X....] duration: 11.898787481s FAIL (2025-07-09 01:25:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.898817648s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 518.750233ms FAIL (2025-07-09 01:24:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== errs: <nil> Jul 9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments... Jul 9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil> Jul 9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload Jul 9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload Jul 9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments... Jul 9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false Jul 9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment... Jul 9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment. Jul 9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment Jul 9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments... Jul 9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection Jul 9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection Jul 9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection Jul 9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments... Jul 9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 518.778155ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....X...X.........X............X.........XXX........X..X...] duration: 5.268551042s FAIL (2025-07-09 01:24:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.268583647s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [..XX...X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..XX] duration: 1m0.000553119s FAIL (2025-07-09 01:24:23 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000567707s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [......X...X.........X............X.........XXX........X..X..] duration: 793.345569ms FAIL (2025-07-09 01:23:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 793.37771ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.168549017s FAIL (2025-07-09 01:23:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== errs: <nil> Jul 9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments... Jul 9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil> Jul 9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload Jul 9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload Jul 9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments... Jul 9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false Jul 9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment... Jul 9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment. Jul 9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment Jul 9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments... Jul 9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection Jul 9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection Jul 9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection Jul 9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments... Jul 9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 1.168586109s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [...XX...X.XXX...X.....XX..X....XX..X.......X....XX..X..XX..X] duration: 1m0.001449721s FAIL (2025-07-09 01:23:23 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001500275s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X......X...X.........X............X.........XXX........X..X.] duration: 1.738138324s FAIL (2025-07-09 01:22:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.738153134s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 428.193555ms FAIL (2025-07-09 01:22:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== errs: <nil> Jul 9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments... Jul 9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil> Jul 9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload Jul 9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload Jul 9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments... Jul 9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false Jul 9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment... Jul 9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment. Jul 9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece... Jul 9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment Jul 9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments... Jul 9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117 , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590 , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010 , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048 , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190 , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299 , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937 , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227 , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662 , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805 , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155 , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911 , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil> Jul 9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078 , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection Jul 9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804 , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223 , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection Jul 9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150 , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection Jul 9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments... Jul 9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 428.29361ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)