Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.623339243s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:14:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.623437025s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.63870272s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:13:38 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.638800052s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.491537671s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:12:41 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.491627528s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.429626095s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:11:27 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.429779278s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.486137766s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:11:02 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.486167727s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.581343846s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:10:05 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.581393065s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.587502085s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:08:58 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.587533968s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.49875087s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:07:54 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.4987832s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.528572127s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:06:57 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.528599945s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.55754949s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:06:12 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.557580379s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.493009193s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:05:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.49303794s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.681830491s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:05:20 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.68193461s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.636872129s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:04:51 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.636918532s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.557126093s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:04:09 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.557152714s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.750094681s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:03:01 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.750129479s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.616724924s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:01:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.616751789s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.564253067s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:01:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.564322454s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.622248712s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 11:00:51 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.62227482s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.578644871s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:59:59 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.578676403s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.542525539s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:59:04 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.542559735s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.446903851s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:57:54 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.446936214s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.439439422s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:56:57 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.439480118s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.549535934s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:56:40 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.549672971s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.589100725s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:55:58 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.589162127s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.479744341s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:55:32 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.479772926s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.547661112s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:54:38 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.54769161s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.456596121s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:54:14 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.456627109s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.668407729s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:53:40 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.668502673s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.601839579s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:52:52 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.601866677s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.582958386s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:52:09 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.583047059s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.514515871s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:51:32 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.514548223s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.514282157s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:50:50 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.514390429s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.67961397s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:50:17 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.679645593s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.556177844s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:49:17 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.556215712s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.564191106s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:48:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.564221146s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.621721518s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:48:25 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.621747918s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.619251454s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:47:44 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.619329777s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.44923588s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:46:31 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 2.449267336s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1.642163953s trace:
https://jaeger.dev.storj.io/trace/233d917505a2379b
FAIL (2026-03-18 10:45:22 UTC)
orbiter warning: operation: Commit uplink: metaclient: context canceled
Duration: 1.642187943s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
474.096953ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:38:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 474.187515ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
406.175267ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:36:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 406.210616ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
393.378481ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:36:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 393.394415ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
496.121236ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:35:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 496.146006ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
395.155091ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:34:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 395.181654ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
424.484662ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:33:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 424.51529ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
502.817511ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:33:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 502.848288ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
562.756911ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:32:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 562.857647ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
512.065779ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:32:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 512.091947ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
485.842086ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:31:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 485.865262ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
523.786209ms trace:
https://jaeger.dev.storj.io/trace/30cb71a3a0ea3f7c
FAIL (2026-03-17 20:30:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 17 19:52:30.637672587 (272h52m28.23444362s) [upload=305639]: Starting upload
Mar 17 19:52:30.637776344 (272h52m28.234547389s) [upload=305639]: Done starting upload
Mar 17 19:52:30.637828070 (272h52m28.234599085s) [upload=305639]: Uploading segments...
Mar 17 19:52:30.638103468 (272h52m28.234874492s) [upload=305639]: Got next segment. Inline: false
Mar 17 19:52:30.638158172 (272h52m28.234929195s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 17 19:52:30.638164003 (272h52m28.234935028s) [upload=305639]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 17 19:52:30.667368502 (272h52m28.264139528s) [seg_pos={0 0}, upload=305639]: Begin upload segment...
Mar 17 19:52:30.667434484 (272h52m28.264205496s) [seg_pos={0 0}, upload=305639]: Done begin upload segment.
Mar 17 19:52:30.667465151 (272h52m28.264236162s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.667564348 (272h52m28.264335382s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668205231 (272h52m28.264976245s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668227975 (272h52m28.264999009s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668804986 (272h52m28.265576001s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.668912306 (272h52m28.265683341s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669319958 (272h52m28.266090985s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669396206 (272h52m28.26616747s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669803576 (272h52m28.266574592s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.669876713 (272h52m28.266647747s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670237889 (272h52m28.267008916s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670397157 (272h52m28.267168191s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670694508 (272h52m28.267465542s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.670870096 (272h52m28.267641127s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.671171129 (272h52m28.267942158s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.672428271 (272h52m28.269199281s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Uploading piece...
Mar 17 19:52:30.777742889 (272h52m28.374513922s) [upload=305639]: Next returned nil segment
Mar 17 19:52:30.777777342 (272h52m28.374548372s) [upload=305639]: Waiting for error group managing segments...
Mar 17 19:52:30.785762386 (272h52m28.38253341s) [noise=true , address=145.79.227.18:4011 , piece_id=3QPPVNVT2TWERDRRZHFXPB4YIWO5JYRXCX22TQ4FWT53HDYUM26Q, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786051171 (272h52m28.382822204s) [noise=true , address=145.79.227.6:4263 , piece_id=7UHC6TAICOCNUNPYKDJKPM4N2XQYMWEFZTSZVTHATGIAK5EKC6SA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786553647 (272h52m28.383324672s) [noise=true , address=145.79.227.42:4127 , piece_id=I7LBWVGOOL2LSIM6E5RKWQ4MTPRK5EU7UKLZODDE7UAQEBPZB2IQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.786856370 (272h52m28.383627403s) [noise=true , address=145.79.227.16:4914 , piece_id=2AMUNXX7LXERIBNWV2B75QZEQ7VRUOSI56WJZMXPC25I352Z34WQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787121414 (272h52m28.383892444s) [noise=true , address=145.79.227.15:4877 , piece_id=VQINGD43LTYZ7YYIYJPAM4MYOICKASF7DHVDAJXBKIRYXFSCYHNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787750176 (272h52m28.384521203s) [noise=true , address=79.127.219.46:4892 , piece_id=GFMQDS6LE4XECHXBLMQNUDH2VSZG4ESBSJBJGERLNIXSJQEUJMPA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.787765631 (272h52m28.384536667s) [noise=true , address=77.90.31.10:4519 , piece_id=HTSNB35LAHFBH5PS7UIZJKL5JCGQQEPDSJYWHV5QXNVDD64YLDLA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.788392891 (272h52m28.385163918s) [noise=true , address=145.79.227.36:4517 , piece_id=GQBOK24CUZXPWIDCAHN34NFZOAMKRWIZO2BT3PJKCYPPTPQRFREA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.802941378 (272h52m28.399712412s) [noise=true , address=145.79.227.52:4447 , piece_id=6WZVLELIKF4YQ32XLHT3CUDHMR3VXHZRLXKOBQE3FSP2DFQO6QHA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.803274113 (272h52m28.400045141s) [noise=true , address=145.79.227.10:4402 , piece_id=4UUASEF764GSGRDUFIJ44QBENUMNU6CWSR7B7I7SMLCB3I5MKQKQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.805132157 (272h52m28.401903183s) [noise=true , address=145.79.227.46:4602 , piece_id=6M2L2E6X32FDJBIMF65EE7W57MR7P7F44Y75ID6XHIE54LCAJMNA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:52:30.811397728 (272h52m28.408168751s) [noise=true , address=145.79.227.25:4621 , piece_id=NXU5I7DB5F5RW7PVRID5O7TY2OFL27HZFJFZIGGKKCF3DJZEOEZQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: <nil>
Mar 17 19:53:00.598803648 (272h52m58.19557468s) [noise=true , address=45.91.116.7:4019 , piece_id=KQII4DOH5TLT72B7YV43JKP5UXKOIQXOTQHVVH7LHWKVODFCTMOQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1ywPSyHckbvqjYJtFQSSZ5aWQKuZV81itg1gqeUunYYZfjfE4D, address:45.91.116.7:4019): protocol: rpc: dial tcp 45.91.116.7:4019: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.598883626 (272h52m58.195654659s) [noise=true , address=45.91.116.8:4080 , piece_id=SJSDVG6WAKV4KCAINAVERFJO7YMF3CLTAFQRONXX2H3PCZZ53XVQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN, address:45.91.116.8:4080): protocol: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33212->45.91.116.8:4080: use of closed network connection
Mar 17 19:53:00.599331932 (272h52m58.196102958s) [noise=true , address=45.91.116.5:4332 , piece_id=TGNPEU6UO4HKTSL3CH447XCAXSTZKXSBZQJ5U5ML6AMCOE47ITRQ, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:1JBXGbBeXrAvFpBao6RroNwiAP4wk4wWWkhAaCcUUatGa2k9Ev, address:45.91.116.5:4332): protocol: write tcp 159.223.143.39:55500->45.91.116.5:4332: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599478080 (272h52m58.196249114s) [noise=true , address=45.91.116.14:4193 , piece_id=6KHHSVRQP77THI7XRFTCRID5AMFHWXTM6HNJRPQWN6NLBG4H5MZA, seg_pos={0 0}, upload=305639]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: rpc: dial tcp 45.91.116.14:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 17 19:53:00.599549317 (272h52m58.196320328s) [seg_pos={0 0}, upload=305639]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 17 19:53:00.633950426 (272h52m58.230721455s) [upload=305639]: Done uploading segments...
Mar 17 19:53:00.633997222 (272h52m58.230768253s) [upload=305639]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 523.817704ms
core
uplink
us-select-1
|