Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
4.191290475s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:18:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.191330964s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
4.327917356s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:17:13 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.327945833s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
10.195526037s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:16:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 10.195553166s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
4.420781478s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:15:49 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.420809757s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
4.772032982s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:15:00 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.772064102s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
4.575714556s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:14:27 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.575739079s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
4.345581948s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:14:05 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.345611553s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
4.568144971s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:13:05 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.568167318s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
4.427173728s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:12:21 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.427197346s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
4.577143733s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:11:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.577169309s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
4.547060849s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:10:49 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.54708638s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
4.428074715s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:09:59 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.428106241s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
4.401879783s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:09:32 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.401905297s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
4.800527643s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:09:08 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.800558879s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
4.571959663s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:08:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.571988339s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
4.45192627s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:07:20 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.451954974s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
4.583004503s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:06:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.583035005s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.064386059s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:05:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.064412944s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.724815415s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:04:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.724844137s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
4.332591826s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:03:51 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.332624015s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
4.415226548s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:03:32 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.415289625s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
4.615544458s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:03:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.615570099s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
4.258418022s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:02:03 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.258449648s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
4.479980676s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:01:45 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.480014731s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
4.538223209s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:01:29 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.538258897s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.063373203s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:00:44 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.063406318s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
4.466057256s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:59:37 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.466081971s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
4.217218067s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:59:05 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.217244788s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
4.455706528s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:58:50 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.455737204s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
21.267138491s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:58:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 21.267169449s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
4.416666571s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:57:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.416692165s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.190337705s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:55:56 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.190366815s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
4.435555878s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:54:47 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.435583383s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.739973588s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:53:57 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.740061819s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
20.552394874s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:53:07 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 20.55242412s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
4.605600251s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:51:59 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.605625148s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
4.512556746s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:50:45 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.512586879s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.550189241s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:50:29 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.55022199s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.537462807s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:49:26 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.537479991s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
4.445638331s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:48:14 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.44566591s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
507.451435ms trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:47:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec 8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec 8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec 8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec 8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec 8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec 8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec 8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec 8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec 8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec 8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 507.477164ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.057153656s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:47:07 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.057179857s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
417.321739ms trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:46:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec 8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec 8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec 8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec 8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec 8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec 8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec 8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec 8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec 8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec 8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 417.344922ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
4.419483436s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:45:56 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.419512364s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
501.592582ms trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:45:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec 8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec 8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec 8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec 8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec 8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec 8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec 8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec 8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec 8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec 8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 501.6506ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
4.357362403s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:45:33 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.357396956s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.393115746s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:45:18 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.393146786s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
520.782347ms trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:45:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec 8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec 8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec 8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec 8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec 8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec 8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec 8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec 8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec 8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec 8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 520.822222ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
404.417147ms trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:44:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec 8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec 8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec 8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec 8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec 8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec 8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec 8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628 , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589 , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877 , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec 8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec 8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257 , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765 , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510 , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654 , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907 , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941 , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583 , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726 , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547 , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546 , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690 , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086 , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec 8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627 , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec 8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec 8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 404.517792ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
8.158711403s trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:44:28 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 8.158725497s
core
uplink
ap1
|