Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 480.943142ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:52:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 480.96788ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 495.073089ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:51:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 495.098961ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 455.219287ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:50:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 455.257983ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 453.35947ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:50:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 453.380143ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 481.014016ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:49:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 481.034246ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 504.764951ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:48:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 504.790876ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 476.356392ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:47:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.381342ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 547.33944ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:46:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 547.365052ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 433.496415ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:45:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 433.528734ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 490.090255ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:44:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 490.117566ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 574.709826ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:44:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 574.73751ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 502.399551ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:43:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 502.442897ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 451.616846ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:42:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.641028ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 458.580391ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:41:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 458.607056ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 456.28923ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:40:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 456.341787ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 440.674105ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:39:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 440.698875ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 464.041543ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:38:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 464.05727ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 492.244535ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:38:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 492.273961ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 461.847993ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:37:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 461.879509ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 485.471887ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:36:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 485.497242ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 468.998412ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:36:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 469.082734ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 508.383838ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:35:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 508.415905ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 486.487492ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:34:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 486.722522ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 437.654701ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:33:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 437.675385ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 476.73808ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:33:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.761778ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 502.8113ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:32:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 502.842694ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 462.62021ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:30:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 462.634628ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 504.342731ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:29:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 504.376195ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 468.573209ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:28:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 468.596049ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 429.266584ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:27:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 429.307047ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 452.189724ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:27:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 452.222847ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 524.951989ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:26:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 524.974214ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 429.117459ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:26:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 429.208476ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 440.771751ms trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:25:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 440.825341ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002358363s trace: https://jaeger.dev.storj.io/trace/5a7d9e8abb1cbd8e FAIL (2025-12-04 07:25:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002382291s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.978545232s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:06:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.978570765s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.420008846s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:05:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.420035059s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.461117019s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:05:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.461145014s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.503652454s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:04:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.503679025s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.438427975s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:04:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.438531033s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.025604042s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:03:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.02569156s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.466552374s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:02:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.46658766s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.617452209s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:01:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.617475714s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.095555725s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 01:00:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.095586535s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.468943144s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 00:59:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.468973692s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.418739322s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 00:59:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.418760217s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.031436706s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 00:58:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.031470428s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.98298776s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 00:57:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 1.983023197s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.023313662s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 00:56:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 2.023342881s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.982545986s trace: https://jaeger.dev.storj.io/trace/7789501b0117be9e FAIL (2025-12-04 00:55:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=564822]: Uploading piece... Dec 4 00:24:55.490490971 (542h38m20.258174404s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490758150 (542h38m20.258441582s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.490867623 (542h38m20.25855105s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491164641 (542h38m20.258848057s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491275804 (542h38m20.258959233s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491509457 (542h38m20.259192874s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.491934674 (542h38m20.259618107s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492243019 (542h38m20.25992643s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492540538 (542h38m20.260223966s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.492833870 (542h38m20.260517295s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Uploading piece... Dec 4 00:24:55.909815998 (542h38m20.677499425s) [upload=564822]: Next returned nil segment Dec 4 00:24:55.909832035 (542h38m20.677515471s) [upload=564822]: Waiting for error group managing segments... Dec 4 00:24:56.086544111 (542h38m20.85422754s) [noise=true , address=81.171.14.70:4030 , piece_id=GKHZYL2VWVGRNQMZEJTMFS33MCR3VUKOV5NWERI6U7MMJKNNVNMQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.097321894 (542h38m20.865005319s) [noise=true , address=185.167.204.206:4070 , piece_id=2EN2H2EY5ELJKV2Q6AONP7W4F6O4ADBCMCRNQUZ3ECCTG6ZKUNJQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098346040 (542h38m20.866029469s) [noise=true , address=81.171.14.70:4029 , piece_id=KP4ATSCNLV7DK4JMHY35BBWS7XCIWFMKF5BVHK4NPI7XD7CF2FPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.098620616 (542h38m20.866304041s) [noise=true , address=185.167.204.206:4048 , piece_id=YOSDWDEUTPARPZW5LQECZ5DW774RG7EI7QNNTGWOJPH2P3ETYNHA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.099631498 (542h38m20.867314938s) [noise=true , address=185.167.204.205:4029 , piece_id=MAZCZWPUBGCBMO54FL7FKOAEWMCQHXQJW37LOJMH6HJD4ZA5DBCQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.102366357 (542h38m20.870049786s) [noise=true , address=185.167.204.205:4017 , piece_id=CMXUL4QT6GWDF7IW5MZCVOP36QJ3YYD64AO2GCOFVHOIUYD3AMBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106095516 (542h38m20.873778943s) [noise=true , address=81.171.14.70:4035 , piece_id=TXFXWMHFCUKLLCO35UW4LLOGXOR645O7R7RINFMVGISIEHI27RLA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.106514180 (542h38m20.874197617s) [noise=true , address=185.167.204.205:4034 , piece_id=3LV6T744IXPCROH32W67XCIKZEVY3V2IMBFCZ3XYNJVWKTEQ4I5A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.107962195 (542h38m20.875645622s) [noise=true , address=185.167.204.205:4011 , piece_id=OSEOGUZKP53P2XOQNECWKNYAP6Q23CKDAJTHQK2G3LFCLSSSXAWQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.108528413 (542h38m20.87621183s) [noise=true , address=185.167.204.205:4027 , piece_id=PUCNTCQA22YU5CV4JCVZ5A6D76RT25XRWVMPLDC6MM4JOZZ63WPA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112620299 (542h38m20.880303725s) [noise=true , address=185.167.204.205:4004 , piece_id=LCAVEBKXUZ6BBWVTU7JD4YWYPC377PJYQLJ6UDUUNYB2443QMWBA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.112746562 (542h38m20.880429978s) [noise=true , address=81.171.14.70:4039 , piece_id=YPGPA2FZHHZVWJIY45U467CMDACNF4D74USOCC6KKBYRY2T5CYIA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.113461832 (542h38m20.881145261s) [noise=true , address=185.167.204.205:4036 , piece_id=HOJWPP7RXPAYKGNPYMABOCRYIDQHKXQZOGM3T6QT6QLAUHXF34WQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117106183 (542h38m20.884789608s) [noise=true , address=185.167.204.206:4061 , piece_id=CUVQ4VTYLQJ4WJTIKX4WG5OEJY5FWDXJ7XZFHQW4GRHYBW7NJWVA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.117791379 (542h38m20.885474804s) [noise=true , address=89.149.201.147:4002 , piece_id=IMR4IBTHU3SHVRX22IHW4BKXUOYFUCOXVL2XR6QGZAVXBU3QMORQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.118638513 (542h38m20.886321941s) [noise=true , address=81.171.14.70:4046 , piece_id=LRTMLBZYXJAINJLAOS4W2J2NOVB3YKUQY5PPEJLP2LIY32OLHBEA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128203460 (542h38m20.895886887s) [noise=true , address=185.167.204.206:4038 , piece_id=YVUYGVOCQ4FTVC35VDKKWL22GLLHF4OJV54FVVBOUKNR26RI5U7A, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.128412152 (542h38m20.896095641s) [noise=true , address=185.167.204.205:4021 , piece_id=6LUIMZKAJES3DIFNUYBIFJIRHPR6DGFBE3NUBNDPJYNCBF3PN6RQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.141262846 (542h38m20.908946272s) [noise=true , address=185.167.204.206:4047 , piece_id=4KDLWARDCIDUNAIB27NMHU2G4QIKURNRHSYOCHQU33KNLXPLR2OA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142453398 (542h38m20.910136823s) [noise=true , address=81.171.14.70:4033 , piece_id=MLPOQYHXDLSPK75A7FPPQW42L3UW6RQMV3NTUBVSMJHI7ZRGDNVQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.142788894 (542h38m20.910472319s) [noise=true , address=185.167.204.205:4028 , piece_id=IFZIBABQFHFWXCUBVP7Y4VMIPWS2SKRULOMYKG4QQY3HZZ7IM2UQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.146866556 (542h38m20.914549981s) [noise=true , address=185.167.204.206:4039 , piece_id=BLX4WSWHF2AXKLZIGGQLAIBYXUWV3R32SAWE4BO6ZOA4XBF5GFCA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.148214588 (542h38m20.915898013s) [noise=true , address=89.149.201.147:4012 , piece_id=3URC2QMMDL7RCPEZ3NE4I5P3XUASXNZ7NAQG5WVEP3GZ5AYT2YKQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.169805920 (542h38m20.937489344s) [noise=true , address=185.167.204.205:4019 , piece_id=IXSVGUCON4YWUI7WRUF6FRLLMTFQ5QXT7THVMNWJDL3AZCABXRUQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.178232046 (542h38m20.945915488s) [noise=true , address=185.167.204.205:4033 , piece_id=6WVC4W2I2CBLMCUUM344FPMFWJ6JO2E3PSUEQVUHMKZF2C3CLDFQ, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181178264 (542h38m20.948861689s) [noise=true , address=81.171.14.70:4025 , piece_id=5ABHTETXY2OWDL5ZBIEFPLY2DJCZYYK4LJAITMG5AH4ZEKUNK2BA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: <nil> Dec 4 00:24:56.181195200 (542h38m20.948878625s) [seg_pos={0 0}, upload=564822]: Segment reached optimal threshold of 26 pieces. Dec 4 00:24:56.181337966 (542h38m20.949021396s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.181344755 (542h38m20.949028179s) [noise=true , address=81.171.14.70:4031 , piece_id=QTJ5PIXVPKBVX7Z7ACMFRXBN3UVJ3SR4A3SATE3SN3KMQJQT6UUA, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186066863 (542h38m20.953750294s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Dec 4 00:24:56.186080458 (542h38m20.953763886s) [noise=true , address=81.171.14.70:4037 , piece_id=IGNVXMRHGZ2Z2AORACJYWTU3FOZ4ETPR5L5ZO5IZNI6GTK56A76Q, seg_pos={0 0}, upload=564822]: Long tail cancellation due to optimal threshold Dec 4 00:24:56.186099573 (542h38m20.953782984s) [seg_pos={0 0}, upload=564822]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 4 00:24:56.186171474 (542h38m20.953854899s) [upload=564822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 4 00:24:56.186175975 (542h38m20.9538594s) [upload=564822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 4 00:24:56.334112694 (542h38m21.101796121s) [upload=564822]: Done uploading segments... Dec 4 00:24:56.334133682 (542h38m21.101817107s) [upload=564822]: Upload finished. err: <nil> Duration: 1.982568875s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)