Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.52076102s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:42:50 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.520790321s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.489921936s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:42:08 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.489949802s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.589225433s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:41:30 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.589252286s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.523814363s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:41:13 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.523840863s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.574571234s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:40:18 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.574616669s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.524170723s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:39:15 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.524199071s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.520953548s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:38:49 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.520989285s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.552786498s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:37:39 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.552816308s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.589163023s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:37:07 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.589200485s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.527841783s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:36:06 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.527890983s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.703565949s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:35:16 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.703599045s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.627146294s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:34:57 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.627175729s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.614478494s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:34:31 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.614513163s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.582907529s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:33:39 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.582935762s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.553780506s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:33:08 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.553855881s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.585692277s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:32:35 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.58571915s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.56440582s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:32:03 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.564437549s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.591073067s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:30:49 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.591121282s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.654822598s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:29:56 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.654926719s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.536465547s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:29:38 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.536493721s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.499781912s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:29:11 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.499812812s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.638771907s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:28:43 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.638797931s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.616086073s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:27:38 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.616138181s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.537881186s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:26:37 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.537905744s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.539030463s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:25:59 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.539060843s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.610113517s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:24:56 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.610197739s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.797704675s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:24:39 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.797731077s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.515305254s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:23:41 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.515339138s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.733808402s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:23:02 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.733858086s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.799226369s trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:22:28 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 2.799257193s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
267.265068ms trace:
https://jaeger.dev.storj.io/trace/33773e65cc9d8d0
FAIL (2026-03-25 03:21:59 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context canceled
Duration: 267.287466ms
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.290729895s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:52:07 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.290765021s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.216982885s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:51:09 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.217006577s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.288434668s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:50:13 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.288465987s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.278272846s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:49:46 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.278313311s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.275808445s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:49:24 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.275846778s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.237582758s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:48:38 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.237610927s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.479110317s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:48:01 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.479158762s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.254042468s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:47:06 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.254086147s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.190185704s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:46:37 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.190217948s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.237154656s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:45:24 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.23718485s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
1.069547366s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:44:14 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.06957496s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.302664291s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:43:33 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.302840924s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
1.226514514s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:43:15 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.226544259s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.54597173s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:42:09 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.546068703s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.318535439s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:41:29 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.318560608s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
1.248789988s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:40:22 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.248822862s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
935.716972ms trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:39:22 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 935.832719ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.225198935s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:38:56 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.225226018s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.220517908s trace:
https://jaeger.dev.storj.io/trace/74532d121d1f43a
FAIL (2026-03-25 01:38:21 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload
Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload
Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments...
Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false
Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment...
Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment.
Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece...
Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment
Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments...
Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil>
Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces
Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF
Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold
Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments...
Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil>
Duration: 1.220609676s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|