Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 243.785811ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:51:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 243.807415ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 211.191563ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:50:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 211.216461ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 222.111715ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:49:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 222.133745ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 223.813258ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:49:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 223.843353ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 235.869505ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:48:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 235.906558ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 221.999135ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:47:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 222.028057ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 219.751362ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:47:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 219.772199ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 216.981796ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:46:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 217.007265ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 239.958081ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:45:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 239.982954ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 224.919327ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:44:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 224.940106ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 231.783205ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:43:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 231.812223ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 265.183994ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:43:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 265.202034ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 241.795157ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:42:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 241.818378ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 244.219942ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:41:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 244.242477ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 220.379067ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:40:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 220.400025ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 250.597786ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:39:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 250.614678ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 240.151521ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:39:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 240.179778ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 231.605817ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:38:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 231.636012ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 224.457258ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:36:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 224.497587ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 225.936148ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:36:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 225.963721ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 218.524447ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:36:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 218.547682ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 247.471014ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:35:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 247.498415ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 236.670952ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:34:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 236.695564ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 242.140536ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:34:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 242.163137ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 225.121635ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:33:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 225.152375ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 228.495078ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:32:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 228.518669ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 230.014977ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:31:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 230.052754ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 229.368474ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:30:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 229.395012ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 244.745797ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:30:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 244.771048ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 238.860163ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:29:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 238.880234ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 246.963912ms trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:29:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 246.987255ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001501679s trace: https://jaeger.dev.storj.io/trace/49c5fc8384c9b05d FAIL (2025-12-01 04:29:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001512738s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 522.954017ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:45:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 522.984115ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 348.259672ms trace: https://jaeger.dev.storj.io/trace/271f991f86732ab4 FAIL (2025-11-30 10:45:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Nov 30 10:01:25.656387382 (456h14m50.424070817s) [upload=474842]: Starting upload Nov 30 10:01:25.656445271 (456h14m50.424128735s) [upload=474842]: Done starting upload Nov 30 10:01:25.656560890 (456h14m50.424244307s) [upload=474842]: Uploading segments... Nov 30 10:01:25.656755948 (456h14m50.424439364s) [upload=474842]: Got next segment. Inline: false Nov 30 10:01:25.656798796 (456h14m50.424482221s) [upload=474842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 30 10:01:25.656804812 (456h14m50.424488237s) [upload=474842]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 30 10:01:25.687253513 (456h14m50.454936951s) [seg_pos={0 0}, upload=474842]: Begin upload segment... Nov 30 10:01:25.687334953 (456h14m50.45501838s) [seg_pos={0 0}, upload=474842]: Done begin upload segment. Nov 30 10:01:25.687365999 (456h14m50.455049426s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.687395623 (456h14m50.455079046s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688017444 (456h14m50.45570086s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688162091 (456h14m50.455845523s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688900072 (456h14m50.456583509s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.689351445 (456h14m50.457034878s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.691963311 (456h14m50.459646742s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.692615279 (456h14m50.460298707s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.693202436 (456h14m50.460885864s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695446378 (456h14m50.463129804s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695995010 (456h14m50.463678438s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.696672685 (456h14m50.464356113s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697327512 (456h14m50.465010946s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697946312 (456h14m50.465629743s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.698588439 (456h14m50.466271867s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.700099718 (456h14m50.467783148s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.701742223 (456h14m50.469425651s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.703455169 (456h14m50.4711386s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.760996264 (456h14m50.528679675s) [upload=474842]: Next returned nil segment Nov 30 10:01:25.761022140 (456h14m50.528705565s) [upload=474842]: Waiting for error group managing segments... Nov 30 10:01:25.765127932 (456h14m50.532811364s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.765342353 (456h14m50.533025792s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769562426 (456h14m50.537245855s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769651014 (456h14m50.53733444s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.779402836 (456h14m50.547086262s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.781233711 (456h14m50.548917136s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.782297741 (456h14m50.549981166s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.789612013 (456h14m50.557295438s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.366555099 (456h14m51.134238525s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.512842120 (456h14m51.280525552s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:27.624485492 (456h14m52.392168921s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:02:25.613199699 (456h15m50.380883132s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: write tcp 159.223.143.39:42214->77.90.31.17:4786: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613253697 (456h15m50.380937133s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: rpc: dial tcp 77.90.31.14:4695: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613513737 (456h15m50.381197173s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: context deadline exceeded; context deadline exceeded Nov 30 10:02:25.613584374 (456h15m50.381267805s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: rpc: dial tcp 77.90.31.19:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613872327 (456h15m50.381555764s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1Liv9RcEgfjxodPuadijzkcUGgJEy7LfKqHGFhaJCKbcxKkNN8, address:77.90.31.15:4713): protocol: rpc: dial tcp 77.90.31.15:4713: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614203209 (456h15m50.381886636s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: rpc: dial tcp 77.90.31.16:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614515289 (456h15m50.382198723s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614557595 (456h15m50.382241009s) [seg_pos={0 0}, upload=474842]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:25.643219626 (456h15m50.410903052s) [upload=474842]: Done uploading segments... Nov 30 10:02:25.643255911 (456h15m50.410939336s) [upload=474842]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 348.281305ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 519.577709ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:44:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 519.597555ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 373.129028ms trace: https://jaeger.dev.storj.io/trace/271f991f86732ab4 FAIL (2025-11-30 10:44:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Nov 30 10:01:25.656387382 (456h14m50.424070817s) [upload=474842]: Starting upload Nov 30 10:01:25.656445271 (456h14m50.424128735s) [upload=474842]: Done starting upload Nov 30 10:01:25.656560890 (456h14m50.424244307s) [upload=474842]: Uploading segments... Nov 30 10:01:25.656755948 (456h14m50.424439364s) [upload=474842]: Got next segment. Inline: false Nov 30 10:01:25.656798796 (456h14m50.424482221s) [upload=474842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 30 10:01:25.656804812 (456h14m50.424488237s) [upload=474842]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 30 10:01:25.687253513 (456h14m50.454936951s) [seg_pos={0 0}, upload=474842]: Begin upload segment... Nov 30 10:01:25.687334953 (456h14m50.45501838s) [seg_pos={0 0}, upload=474842]: Done begin upload segment. Nov 30 10:01:25.687365999 (456h14m50.455049426s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.687395623 (456h14m50.455079046s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688017444 (456h14m50.45570086s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688162091 (456h14m50.455845523s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688900072 (456h14m50.456583509s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.689351445 (456h14m50.457034878s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.691963311 (456h14m50.459646742s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.692615279 (456h14m50.460298707s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.693202436 (456h14m50.460885864s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695446378 (456h14m50.463129804s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695995010 (456h14m50.463678438s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.696672685 (456h14m50.464356113s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697327512 (456h14m50.465010946s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697946312 (456h14m50.465629743s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.698588439 (456h14m50.466271867s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.700099718 (456h14m50.467783148s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.701742223 (456h14m50.469425651s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.703455169 (456h14m50.4711386s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.760996264 (456h14m50.528679675s) [upload=474842]: Next returned nil segment Nov 30 10:01:25.761022140 (456h14m50.528705565s) [upload=474842]: Waiting for error group managing segments... Nov 30 10:01:25.765127932 (456h14m50.532811364s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.765342353 (456h14m50.533025792s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769562426 (456h14m50.537245855s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769651014 (456h14m50.53733444s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.779402836 (456h14m50.547086262s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.781233711 (456h14m50.548917136s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.782297741 (456h14m50.549981166s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.789612013 (456h14m50.557295438s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.366555099 (456h14m51.134238525s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.512842120 (456h14m51.280525552s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:27.624485492 (456h14m52.392168921s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:02:25.613199699 (456h15m50.380883132s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: write tcp 159.223.143.39:42214->77.90.31.17:4786: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613253697 (456h15m50.380937133s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: rpc: dial tcp 77.90.31.14:4695: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613513737 (456h15m50.381197173s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: context deadline exceeded; context deadline exceeded Nov 30 10:02:25.613584374 (456h15m50.381267805s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: rpc: dial tcp 77.90.31.19:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613872327 (456h15m50.381555764s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1Liv9RcEgfjxodPuadijzkcUGgJEy7LfKqHGFhaJCKbcxKkNN8, address:77.90.31.15:4713): protocol: rpc: dial tcp 77.90.31.15:4713: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614203209 (456h15m50.381886636s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: rpc: dial tcp 77.90.31.16:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614515289 (456h15m50.382198723s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614557595 (456h15m50.382241009s) [seg_pos={0 0}, upload=474842]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:25.643219626 (456h15m50.410903052s) [upload=474842]: Done uploading segments... Nov 30 10:02:25.643255911 (456h15m50.410939336s) [upload=474842]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 373.156357ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 412.185965ms trace: https://jaeger.dev.storj.io/trace/35e31fda89825398 FAIL (2025-11-30 10:44:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 412.208681ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 477.143203ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:43:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 477.171315ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 354.817532ms trace: https://jaeger.dev.storj.io/trace/271f991f86732ab4 FAIL (2025-11-30 10:43:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Nov 30 10:01:25.656387382 (456h14m50.424070817s) [upload=474842]: Starting upload Nov 30 10:01:25.656445271 (456h14m50.424128735s) [upload=474842]: Done starting upload Nov 30 10:01:25.656560890 (456h14m50.424244307s) [upload=474842]: Uploading segments... Nov 30 10:01:25.656755948 (456h14m50.424439364s) [upload=474842]: Got next segment. Inline: false Nov 30 10:01:25.656798796 (456h14m50.424482221s) [upload=474842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 30 10:01:25.656804812 (456h14m50.424488237s) [upload=474842]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 30 10:01:25.687253513 (456h14m50.454936951s) [seg_pos={0 0}, upload=474842]: Begin upload segment... Nov 30 10:01:25.687334953 (456h14m50.45501838s) [seg_pos={0 0}, upload=474842]: Done begin upload segment. Nov 30 10:01:25.687365999 (456h14m50.455049426s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.687395623 (456h14m50.455079046s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688017444 (456h14m50.45570086s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688162091 (456h14m50.455845523s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688900072 (456h14m50.456583509s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.689351445 (456h14m50.457034878s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.691963311 (456h14m50.459646742s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.692615279 (456h14m50.460298707s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.693202436 (456h14m50.460885864s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695446378 (456h14m50.463129804s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695995010 (456h14m50.463678438s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.696672685 (456h14m50.464356113s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697327512 (456h14m50.465010946s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697946312 (456h14m50.465629743s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.698588439 (456h14m50.466271867s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.700099718 (456h14m50.467783148s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.701742223 (456h14m50.469425651s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.703455169 (456h14m50.4711386s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.760996264 (456h14m50.528679675s) [upload=474842]: Next returned nil segment Nov 30 10:01:25.761022140 (456h14m50.528705565s) [upload=474842]: Waiting for error group managing segments... Nov 30 10:01:25.765127932 (456h14m50.532811364s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.765342353 (456h14m50.533025792s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769562426 (456h14m50.537245855s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769651014 (456h14m50.53733444s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.779402836 (456h14m50.547086262s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.781233711 (456h14m50.548917136s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.782297741 (456h14m50.549981166s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.789612013 (456h14m50.557295438s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.366555099 (456h14m51.134238525s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.512842120 (456h14m51.280525552s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:27.624485492 (456h14m52.392168921s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:02:25.613199699 (456h15m50.380883132s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: write tcp 159.223.143.39:42214->77.90.31.17:4786: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613253697 (456h15m50.380937133s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: rpc: dial tcp 77.90.31.14:4695: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613513737 (456h15m50.381197173s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: context deadline exceeded; context deadline exceeded Nov 30 10:02:25.613584374 (456h15m50.381267805s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: rpc: dial tcp 77.90.31.19:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613872327 (456h15m50.381555764s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1Liv9RcEgfjxodPuadijzkcUGgJEy7LfKqHGFhaJCKbcxKkNN8, address:77.90.31.15:4713): protocol: rpc: dial tcp 77.90.31.15:4713: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614203209 (456h15m50.381886636s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: rpc: dial tcp 77.90.31.16:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614515289 (456h15m50.382198723s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614557595 (456h15m50.382241009s) [seg_pos={0 0}, upload=474842]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:25.643219626 (456h15m50.410903052s) [upload=474842]: Done uploading segments... Nov 30 10:02:25.643255911 (456h15m50.410939336s) [upload=474842]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 354.850689ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [XX..........................................................] duration: 395.130907ms trace: https://jaeger.dev.storj.io/trace/35e31fda89825398 FAIL (2025-11-30 10:43:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 395.15612ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [XXX.........................................................] duration: 397.245834ms trace: https://jaeger.dev.storj.io/trace/35e31fda89825398 FAIL (2025-11-30 10:43:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 397.269248ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 502.78076ms trace: https://jaeger.dev.storj.io/trace/271f991f86732ab4 FAIL (2025-11-30 10:43:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Nov 30 10:01:25.656387382 (456h14m50.424070817s) [upload=474842]: Starting upload Nov 30 10:01:25.656445271 (456h14m50.424128735s) [upload=474842]: Done starting upload Nov 30 10:01:25.656560890 (456h14m50.424244307s) [upload=474842]: Uploading segments... Nov 30 10:01:25.656755948 (456h14m50.424439364s) [upload=474842]: Got next segment. Inline: false Nov 30 10:01:25.656798796 (456h14m50.424482221s) [upload=474842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 30 10:01:25.656804812 (456h14m50.424488237s) [upload=474842]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 30 10:01:25.687253513 (456h14m50.454936951s) [seg_pos={0 0}, upload=474842]: Begin upload segment... Nov 30 10:01:25.687334953 (456h14m50.45501838s) [seg_pos={0 0}, upload=474842]: Done begin upload segment. Nov 30 10:01:25.687365999 (456h14m50.455049426s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.687395623 (456h14m50.455079046s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688017444 (456h14m50.45570086s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688162091 (456h14m50.455845523s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688900072 (456h14m50.456583509s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.689351445 (456h14m50.457034878s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.691963311 (456h14m50.459646742s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.692615279 (456h14m50.460298707s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.693202436 (456h14m50.460885864s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695446378 (456h14m50.463129804s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695995010 (456h14m50.463678438s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.696672685 (456h14m50.464356113s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697327512 (456h14m50.465010946s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697946312 (456h14m50.465629743s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.698588439 (456h14m50.466271867s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.700099718 (456h14m50.467783148s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.701742223 (456h14m50.469425651s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.703455169 (456h14m50.4711386s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.760996264 (456h14m50.528679675s) [upload=474842]: Next returned nil segment Nov 30 10:01:25.761022140 (456h14m50.528705565s) [upload=474842]: Waiting for error group managing segments... Nov 30 10:01:25.765127932 (456h14m50.532811364s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.765342353 (456h14m50.533025792s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769562426 (456h14m50.537245855s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769651014 (456h14m50.53733444s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.779402836 (456h14m50.547086262s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.781233711 (456h14m50.548917136s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.782297741 (456h14m50.549981166s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.789612013 (456h14m50.557295438s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.366555099 (456h14m51.134238525s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.512842120 (456h14m51.280525552s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:27.624485492 (456h14m52.392168921s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:02:25.613199699 (456h15m50.380883132s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: write tcp 159.223.143.39:42214->77.90.31.17:4786: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613253697 (456h15m50.380937133s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: rpc: dial tcp 77.90.31.14:4695: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613513737 (456h15m50.381197173s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: context deadline exceeded; context deadline exceeded Nov 30 10:02:25.613584374 (456h15m50.381267805s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: rpc: dial tcp 77.90.31.19:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613872327 (456h15m50.381555764s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1Liv9RcEgfjxodPuadijzkcUGgJEy7LfKqHGFhaJCKbcxKkNN8, address:77.90.31.15:4713): protocol: rpc: dial tcp 77.90.31.15:4713: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614203209 (456h15m50.381886636s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: rpc: dial tcp 77.90.31.16:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614515289 (456h15m50.382198723s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614557595 (456h15m50.382241009s) [seg_pos={0 0}, upload=474842]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:25.643219626 (456h15m50.410903052s) [upload=474842]: Done uploading segments... Nov 30 10:02:25.643255911 (456h15m50.410939336s) [upload=474842]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 502.802619ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 482.430708ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:42:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 482.472043ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 490.201868ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:42:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 490.264744ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [XXXX........................................................] duration: 386.299258ms trace: https://jaeger.dev.storj.io/trace/35e31fda89825398 FAIL (2025-11-30 10:42:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 386.320258ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.XXXXX......................................................] duration: 677.746652ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:42:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 677.775013ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 396.461641ms trace: https://jaeger.dev.storj.io/trace/271f991f86732ab4 FAIL (2025-11-30 10:41:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Nov 30 10:01:25.656387382 (456h14m50.424070817s) [upload=474842]: Starting upload Nov 30 10:01:25.656445271 (456h14m50.424128735s) [upload=474842]: Done starting upload Nov 30 10:01:25.656560890 (456h14m50.424244307s) [upload=474842]: Uploading segments... Nov 30 10:01:25.656755948 (456h14m50.424439364s) [upload=474842]: Got next segment. Inline: false Nov 30 10:01:25.656798796 (456h14m50.424482221s) [upload=474842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 30 10:01:25.656804812 (456h14m50.424488237s) [upload=474842]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 30 10:01:25.687253513 (456h14m50.454936951s) [seg_pos={0 0}, upload=474842]: Begin upload segment... Nov 30 10:01:25.687334953 (456h14m50.45501838s) [seg_pos={0 0}, upload=474842]: Done begin upload segment. Nov 30 10:01:25.687365999 (456h14m50.455049426s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.687395623 (456h14m50.455079046s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688017444 (456h14m50.45570086s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688162091 (456h14m50.455845523s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688900072 (456h14m50.456583509s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.689351445 (456h14m50.457034878s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.691963311 (456h14m50.459646742s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.692615279 (456h14m50.460298707s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.693202436 (456h14m50.460885864s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695446378 (456h14m50.463129804s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695995010 (456h14m50.463678438s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.696672685 (456h14m50.464356113s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697327512 (456h14m50.465010946s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697946312 (456h14m50.465629743s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.698588439 (456h14m50.466271867s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.700099718 (456h14m50.467783148s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.701742223 (456h14m50.469425651s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.703455169 (456h14m50.4711386s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.760996264 (456h14m50.528679675s) [upload=474842]: Next returned nil segment Nov 30 10:01:25.761022140 (456h14m50.528705565s) [upload=474842]: Waiting for error group managing segments... Nov 30 10:01:25.765127932 (456h14m50.532811364s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.765342353 (456h14m50.533025792s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769562426 (456h14m50.537245855s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769651014 (456h14m50.53733444s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.779402836 (456h14m50.547086262s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.781233711 (456h14m50.548917136s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.782297741 (456h14m50.549981166s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.789612013 (456h14m50.557295438s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.366555099 (456h14m51.134238525s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.512842120 (456h14m51.280525552s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:27.624485492 (456h14m52.392168921s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:02:25.613199699 (456h15m50.380883132s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: write tcp 159.223.143.39:42214->77.90.31.17:4786: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613253697 (456h15m50.380937133s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: rpc: dial tcp 77.90.31.14:4695: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613513737 (456h15m50.381197173s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: context deadline exceeded; context deadline exceeded Nov 30 10:02:25.613584374 (456h15m50.381267805s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: rpc: dial tcp 77.90.31.19:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613872327 (456h15m50.381555764s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1Liv9RcEgfjxodPuadijzkcUGgJEy7LfKqHGFhaJCKbcxKkNN8, address:77.90.31.15:4713): protocol: rpc: dial tcp 77.90.31.15:4713: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614203209 (456h15m50.381886636s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: rpc: dial tcp 77.90.31.16:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614515289 (456h15m50.382198723s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614557595 (456h15m50.382241009s) [seg_pos={0 0}, upload=474842]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:25.643219626 (456h15m50.410903052s) [upload=474842]: Done uploading segments... Nov 30 10:02:25.643255911 (456h15m50.410939336s) [upload=474842]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 396.487952ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..XXXXX.....................................................] duration: 504.778962ms trace: https://jaeger.dev.storj.io/trace/1075f18a004aadd1 FAIL (2025-11-30 10:41:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 504.806686ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [XXXXX.......................................................] duration: 383.514749ms trace: https://jaeger.dev.storj.io/trace/35e31fda89825398 FAIL (2025-11-30 10:41:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 383.566562ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.XXXXX......................................................] duration: 364.276433ms trace: https://jaeger.dev.storj.io/trace/271f991f86732ab4 FAIL (2025-11-30 10:41:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Nov 30 10:01:25.656387382 (456h14m50.424070817s) [upload=474842]: Starting upload Nov 30 10:01:25.656445271 (456h14m50.424128735s) [upload=474842]: Done starting upload Nov 30 10:01:25.656560890 (456h14m50.424244307s) [upload=474842]: Uploading segments... Nov 30 10:01:25.656755948 (456h14m50.424439364s) [upload=474842]: Got next segment. Inline: false Nov 30 10:01:25.656798796 (456h14m50.424482221s) [upload=474842]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 30 10:01:25.656804812 (456h14m50.424488237s) [upload=474842]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 30 10:01:25.687253513 (456h14m50.454936951s) [seg_pos={0 0}, upload=474842]: Begin upload segment... Nov 30 10:01:25.687334953 (456h14m50.45501838s) [seg_pos={0 0}, upload=474842]: Done begin upload segment. Nov 30 10:01:25.687365999 (456h14m50.455049426s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.687395623 (456h14m50.455079046s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688017444 (456h14m50.45570086s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688162091 (456h14m50.455845523s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.688900072 (456h14m50.456583509s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.689351445 (456h14m50.457034878s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.691963311 (456h14m50.459646742s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.692615279 (456h14m50.460298707s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.693202436 (456h14m50.460885864s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695446378 (456h14m50.463129804s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.695995010 (456h14m50.463678438s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.696672685 (456h14m50.464356113s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697327512 (456h14m50.465010946s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.697946312 (456h14m50.465629743s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.698588439 (456h14m50.466271867s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.700099718 (456h14m50.467783148s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.701742223 (456h14m50.469425651s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.703455169 (456h14m50.4711386s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Uploading piece... Nov 30 10:01:25.760996264 (456h14m50.528679675s) [upload=474842]: Next returned nil segment Nov 30 10:01:25.761022140 (456h14m50.528705565s) [upload=474842]: Waiting for error group managing segments... Nov 30 10:01:25.765127932 (456h14m50.532811364s) [noise=true , address=45.91.116.13:4176 , piece_id=RMHBCMZXZOZ4K7MKKWPBHLUHHS5M4KHJEURCYN267ZWEHKRTFPEQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.765342353 (456h14m50.533025792s) [noise=true , address=45.91.116.12:4230 , piece_id=MQBBHIHFRVVGE55MUD5734Y7DP35KHJYUXVIUG37H6TNLR5KZHOA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769562426 (456h14m50.537245855s) [noise=true , address=145.79.227.18:4010 , piece_id=WM6E5RW7AXH4TL67NTYDQ6DWNLF2BLRPD7FV2RFXHI3ZRZODIHSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.769651014 (456h14m50.53733444s) [noise=true , address=145.79.227.4:4202 , piece_id=YXCIJGVFQIMNTLLTRANXV2GKK5U6J6MQZSRY4ZESZ4GI2GHSF4EQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.779402836 (456h14m50.547086262s) [noise=true , address=45.91.116.5:4322 , piece_id=6TZVML2CVTFYCILQPEVTKYQUUVXB27426VFPXORXAORRODJ4VRKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.781233711 (456h14m50.548917136s) [noise=true , address=145.79.227.12:4786 , piece_id=QLZTONTBXKDDILOIUS4VEZP425CU27C4DG25VGEFUEE26646NFVQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.782297741 (456h14m50.549981166s) [noise=true , address=45.91.116.8:4057 , piece_id=DDLKYEFWA24APBF252PZC6NVJKWM43JJMQF62ION2TAXOQOKGDRA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:25.789612013 (456h14m50.557295438s) [noise=true , address=45.91.116.11:4123 , piece_id=CCP7QZFHM2QBWZTHGM5ZH6DR3YRH2DSBYJG4H2S2L5WRYX4OYVBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.366555099 (456h14m51.134238525s) [noise=true , address=64.49.13.27:4964 , piece_id=RRTDHAKO2ZITDQNMO2YKOKABSLDW3LW5ON2IGT62L4J5KWFH23VQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:26.512842120 (456h14m51.280525552s) [noise=true , address=64.49.13.154:4931 , piece_id=RNIK6MIHYPWENZJCRSMREL7CQDWZQORW3ZQ7H6XJ4VG2U3MGY5LA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:01:27.624485492 (456h14m52.392168921s) [noise=true , address=64.49.13.169:4709 , piece_id=THMOKKNHIAEOOJZSVD6XP3B2N5ZWYAUUKM2F3IINA3QNYBPBUA4A, seg_pos={0 0}, upload=474842]: Done uploading piece. err: <nil> Nov 30 10:02:25.613199699 (456h15m50.380883132s) [noise=true , address=77.90.31.17:4786 , piece_id=ZOGFARJYPMWQRKSBI36CPDHXE2T3ZAJNNER7WI5XI6GBR7FC5GSA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: write tcp 159.223.143.39:42214->77.90.31.17:4786: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613253697 (456h15m50.380937133s) [noise=true , address=77.90.31.14:4695 , piece_id=C2OMKX5PPARTCMVVOYVO6OYLO3UOGV6H5H5K7PAE2UMRXWQ2PIMQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: rpc: dial tcp 77.90.31.14:4695: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613513737 (456h15m50.381197173s) [noise=true , address=77.90.31.10:4548 , piece_id=G2FNV2WFZPRMGZJ7OCYLKNS5WBF6DRDKIGWD2MBA6SBOYNKB7XKQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: context deadline exceeded; context deadline exceeded Nov 30 10:02:25.613584374 (456h15m50.381267805s) [noise=true , address=77.90.31.19:4873 , piece_id=77VCGVGDXVDOHNIG6I7SGKKNUJ4LSXWPOGXLSHGZSN6DWPBHMACA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:122JpURwWNSDakKSHma7tqooTbiTewq6RQcCD8avQX8SKGeRcYV, address:77.90.31.19:4873): protocol: rpc: dial tcp 77.90.31.19:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.613872327 (456h15m50.381555764s) [noise=true , address=77.90.31.15:4713 , piece_id=2DQC2GT255JL622H72TD6TBTIKYIZF22KI6DO4HOUR56I2OZRXPQ, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1Liv9RcEgfjxodPuadijzkcUGgJEy7LfKqHGFhaJCKbcxKkNN8, address:77.90.31.15:4713): protocol: rpc: dial tcp 77.90.31.15:4713: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614203209 (456h15m50.381886636s) [noise=true , address=77.90.31.16:4754 , piece_id=2PZWMXGP2JH5VOML6H5D2DLDBO6RXTVP7AJICZWTWRPGGJ2IOC5Q, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: rpc: dial tcp 77.90.31.16:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614515289 (456h15m50.382198723s) [noise=true , address=77.90.31.11:4564 , piece_id=NVWKGGVW2FZGL3JVESJFXQ6OBUDWC3MSNNAXF7YC7B7ZK72R3EBA, seg_pos={0 0}, upload=474842]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:25.614557595 (456h15m50.382241009s) [seg_pos={0 0}, upload=474842]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:25.643219626 (456h15m50.410903052s) [upload=474842]: Done uploading segments... Nov 30 10:02:25.643255911 (456h15m50.410939336s) [upload=474842]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 364.302704ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)