Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 431.172679ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:36:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 431.202463ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 488.388526ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:35:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 488.415029ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 429.564389ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:34:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 429.589504ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 463.371613ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:34:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 463.400131ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 449.651239ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:34:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 449.673003ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 485.038392ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:33:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 485.064233ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 437.259439ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:32:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 437.288386ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 446.490779ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:31:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 446.530575ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 440.869502ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:30:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 440.912907ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 444.726563ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:29:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 444.754355ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 484.368376ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:29:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 484.395556ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 422.802852ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:28:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 423.250808ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 439.685066ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:28:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 439.72949ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 451.285377ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:27:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.316699ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 454.014164ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:26:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.041786ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 447.397515ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:25:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 447.442935ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 436.579571ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:24:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 436.630056ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 433.615416ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:23:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 433.63141ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 479.785909ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:23:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.87538ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 463.872241ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:22:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 463.904703ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 420.294112ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:20:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 420.329191ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 451.151884ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:20:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.191255ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 440.051257ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:19:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 440.084887ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 435.336107ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:18:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 435.378142ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 426.924416ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:17:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 427.019556ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 464.052944ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:16:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 464.08334ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 462.703934ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:16:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 462.760009ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 486.287089ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:14:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 486.317761ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 474.15933ms trace: https://jaeger.dev.storj.io/trace/32caaa369a21039 FAIL (2026-01-27 05:13:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 474.184081ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 395.667007ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:28:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 395.696156ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 454.395694ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:27:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 454.425612ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 399.309778ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:26:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 399.35053ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 444.615273ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:26:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 444.705298ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 697.603025ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:25:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 697.628798ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 706.731506ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:24:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 706.756765ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.596361434s trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:23:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1.596391246s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 731.196762ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:23:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 731.304736ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 523.213414ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:22:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 523.274244ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 394.265534ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:21:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 394.295271ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 470.738575ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:21:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 470.768775ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 463.812128ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:20:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 463.841677ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 464.240074ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:20:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 464.273611ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 411.624597ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:19:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 411.650341ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 393.54945ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:18:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 393.645308ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 374.205708ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:17:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 374.257339ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 430.962926ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:17:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 430.995813ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 412.666227ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:16:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 412.808131ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 416.673333ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:15:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 416.78743ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 424.505438ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:14:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 424.534465ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 433.682497ms trace: https://jaeger.dev.storj.io/trace/649ff569d9260430 FAIL (2026-01-26 19:13:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Jan 26 18:42:54.892555126 (266h47m48.376012329s) [upload=277402]: Starting upload Jan 26 18:42:54.892618330 (266h47m48.376075534s) [upload=277402]: Done starting upload Jan 26 18:42:54.892806153 (266h47m48.376263351s) [upload=277402]: Uploading segments... Jan 26 18:42:54.893216828 (266h47m48.376674016s) [upload=277402]: Got next segment. Inline: false Jan 26 18:42:54.893361259 (266h47m48.376818446s) [upload=277402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:42:54.893370748 (266h47m48.376827949s) [upload=277402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:42:54.922623624 (266h47m48.406080821s) [seg_pos={0 0}, upload=277402]: Begin upload segment... Jan 26 18:42:54.922705164 (266h47m48.406162361s) [seg_pos={0 0}, upload=277402]: Done begin upload segment. Jan 26 18:42:54.923267340 (266h47m48.406724546s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923295520 (266h47m48.406752723s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.923463714 (266h47m48.406920914s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924171917 (266h47m48.407629114s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.924960809 (266h47m48.408418009s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.925709003 (266h47m48.409166203s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926163685 (266h47m48.409620887s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926544615 (266h47m48.410001814s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.926886671 (266h47m48.410343871s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.927389323 (266h47m48.410846526s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928052040 (266h47m48.411509242s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.928456971 (266h47m48.411914167s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929139732 (266h47m48.412596935s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.929523243 (266h47m48.412980441s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930068115 (266h47m48.413525312s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.930308711 (266h47m48.413765917s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.931658434 (266h47m48.415115739s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.933055168 (266h47m48.416512371s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Uploading piece... Jan 26 18:42:54.996155914 (266h47m48.479613122s) [upload=277402]: Next returned nil segment Jan 26 18:42:54.996173908 (266h47m48.479631113s) [upload=277402]: Waiting for error group managing segments... Jan 26 18:42:55.003483215 (266h47m48.486940403s) [noise=true , address=77.90.31.13:4639 , piece_id=RR6AIMXIKCWBAJB76EVIR7XPTIIXJF6HYWS6K33Z2XXZTQG7W3KA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.012729236 (266h47m48.49618644s) [noise=true , address=77.90.31.18:4820 , piece_id=JHSE3TIAG6IWPZLIOXQOOOR35S3JIFUL5MPARKVMPP5N6GGDOZVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.014803197 (266h47m48.498260385s) [noise=true , address=77.90.31.14:4687 , piece_id=DYYHZWGP5GQXMNV6W6LGEI2RC4OKUVMTGQMJO5BMDOBHZO7SNSBA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.016430206 (266h47m48.499887394s) [noise=true , address=145.79.227.20:4444 , piece_id=C2DCCBK7WKSIF5SDT7H3XSFPOG4SFXBAGUMGZRZZYN5452INR4HQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.018627794 (266h47m48.502084992s) [noise=true , address=145.79.227.16:4932 , piece_id=IYDCWLAAL7F62ANLEDOGEQRD47WWDBLVDXNDUEOHJA4PWPSQWPTA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.029164169 (266h47m48.512621371s) [noise=true , address=77.90.31.17:4779 , piece_id=RGDCMOCINVOWTEUGZRI7NTFJDQ7HRVZPLH33H4NRCWBW6IV4UJKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.036616256 (266h47m48.520073453s) [noise=true , address=145.79.227.7:4318 , piece_id=NKCWSINHKSFOYXDYBEISUJUR6X7L5AFGFH5XCIW7BENQGSQD4UGA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.046925126 (266h47m48.530382315s) [noise=true , address=145.79.227.19:4086 , piece_id=SDYUP5JH7UUCWD6JF2PXV2XJ7EB5XLQDQAXJDWNN6OHSIMUFPGQA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.123244074 (266h47m48.606701261s) [noise=true , address=5.231.73.3:4017 , piece_id=ECPITEORVHTR3O2QI5FU4WFA3QPYPMDY2DQAHGA6SA65NGM7C3ZQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:42:55.662288462 (266h47m49.14574566s) [noise=true , address=64.49.13.137:4501 , piece_id=6JJR7LUIJRHG7VJRXA3CT3RCE2MZ47OHVTASU2I5S5XPQFUBMTWA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: <nil> Jan 26 18:43:54.856814419 (266h48m48.340271626s) [noise=true , address=45.91.116.13:4179 , piece_id=YNMMIVLRMNG4UG6K5G2CN73GGLDRUM766RWAY7TBEXHQKA4SZKVQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12pFjszrBBZvWsoeXM43kLgX7uNqk8iwSG8nsE1zbj77GVfQcKn, address:45.91.116.13:4179): protocol: rpc: dial tcp 45.91.116.13:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857250991 (266h48m48.340708194s) [noise=true , address=45.91.116.6:4999 , piece_id=VCGR7AB3S6MQ6MEXBNZV55IHMQ6N4JEN5S5YLZFSSWSCLYI5MLSQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: rpc: dial tcp 45.91.116.6:4999: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857528940 (266h48m48.340986149s) [noise=true , address=45.91.116.8:4050 , piece_id=JUW3JJDHKIB2A6OM73XPNR5D6LAKEHXGRXEQ5JAYXO3GYYXO6EYQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1WVHPacX35JJ8PFzCs3UtGwsR4yVHXKmiqfGedK9KfXbHNJtmu, address:45.91.116.8:4050): protocol: rpc: dial tcp 45.91.116.8:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.857718675 (266h48m48.341175881s) [noise=true , address=45.91.116.10:4115 , piece_id=MOZKGOXBXSGIG4FLKUIHXMVPVL2FDIQ3NNDJ6B4BCDKCARV4MUDA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1arcuTW5rwnvfiqi2H7dL3oL6jKBkDeGq6DmgRGpS9Vs9BCHmf, address:45.91.116.10:4115): protocol: rpc: dial tcp 45.91.116.10:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858114005 (266h48m48.34157121s) [noise=true , address=45.91.116.4:4966 , piece_id=C3OLIBWH7ZKQE77KNXWACJU7BRWACZTAOV7GLRFY36NKGORO3OKQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S, address:45.91.116.4:4966): protocol: rpc: dial tcp 45.91.116.4:4966: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858332249 (266h48m48.341789455s) [noise=true , address=45.91.116.12:4248 , piece_id=QSE4CAINLY5QGRKAVMT3K55LXTURWRNZEAGN4YCVVPDBX5Q6XLSA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG, address:45.91.116.12:4248): protocol: rpc: dial tcp 45.91.116.12:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858544711 (266h48m48.342001924s) [noise=true , address=45.91.116.5:4309 , piece_id=ENYXWOUB3VRTYWCNQ56M5TWJ64KPBNQUV35WH3L2KSWSLBPG5JHA, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:1CHPuJdsjXtHCho9WdqpSvzRbEJqmAXchTkhmXgVqWQTTTCrV3, address:45.91.116.5:4309): protocol: read tcp 159.223.143.39:59926->45.91.116.5:4309: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858684804 (266h48m48.34214201s) [noise=true , address=45.91.116.14:4195 , piece_id=AYK5G4SDZRLKVSO54LNF3KETZI4S3NLRGU6CQBZXENLEUO53OLDQ, seg_pos={0 0}, upload=277402]: Done uploading piece. err: ecclient: upload failed (node:12N8gL2cPKtuvwn66hcZWQsoe3KRUYgHoy5FTJ38KZstPzzxiKt, address:45.91.116.14:4195): protocol: write tcp 159.223.143.39:55974->45.91.116.14:4195: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:54.858709394 (266h48m48.342166577s) [seg_pos={0 0}, upload=277402]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jan 26 18:43:54.893872512 (266h48m48.377329709s) [upload=277402]: Done uploading segments... Jan 26 18:43:54.893952229 (266h48m48.37740943s) [upload=277402]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 433.714332ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)