Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 164725 reports (Page 142 of 3295)
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
4.191290475s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:18:15 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.191330964s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
4.327917356s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:17:13 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.327945833s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
10.195526037s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:16:46 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 10.195553166s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
4.420781478s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:15:49 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.420809757s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
4.772032982s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:15:00 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.772064102s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
4.575714556s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:14:27 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.575739079s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration:
4.345581948s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:14:05 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.345611553s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................]
duration:
4.568144971s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:13:05 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.568167318s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................]
duration:
4.427173728s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:12:21 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.427197346s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................]
duration:
4.577143733s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:11:34 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.577169309s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................]
duration:
4.547060849s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:10:49 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.54708638s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................]
duration:
4.428074715s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:09:59 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.428106241s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................]
duration:
4.401879783s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:09:32 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.401905297s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................]
duration:
4.800527643s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:09:08 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.800558879s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................]
duration:
4.571959663s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:08:34 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.571988339s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................]
duration:
4.45192627s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:07:20 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.451954974s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................]
duration:
4.583004503s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:06:15 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.583035005s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................]
duration:
5.064386059s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:05:24 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.064412944s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................]
duration:
5.724815415s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:04:15 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.724844137s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration:
4.332591826s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:03:51 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.332624015s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration:
4.415226548s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:03:32 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.415289625s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................]
duration:
4.615544458s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:03:06 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.615570099s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration:
4.258418022s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:02:03 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.258449648s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration:
4.479980676s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:01:45 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.480014731s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration:
4.538223209s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:01:29 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.538258897s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration:
5.063373203s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 18:00:44 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.063406318s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
4.466057256s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:59:37 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.466081971s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
4.217218067s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:59:05 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.217244788s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
4.455706528s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:58:50 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.455737204s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
21.267138491s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:58:34 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 21.267169449s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
4.416666571s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:57:06 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.416692165s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
5.190337705s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:55:56 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.190366815s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
4.435555878s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:54:47 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.435583383s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
4.739973588s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:53:57 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.740061819s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
20.552394874s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:53:07 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 20.55242412s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
4.605600251s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:51:59 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.605625148s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
4.512556746s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:50:45 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.512586879s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
4.550189241s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:50:29 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.55022199s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
5.537462807s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:49:26 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.537479991s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
4.445638331s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:48:14 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.44566591s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................]
duration:
507.451435ms
trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:47:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec  8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec  8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec  8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec  8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec  8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec  8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec  8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec  8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec  8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec  8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec  8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec  8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec  8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec  8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 507.477164ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
5.057153656s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:47:07 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.057179857s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration:
417.321739ms
trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:46:43 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec  8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec  8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec  8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec  8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec  8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec  8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec  8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec  8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec  8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec  8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec  8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec  8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec  8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec  8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 417.344922ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
4.419483436s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:45:56 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.419512364s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration:
501.592582ms
trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:45:37 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec  8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec  8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec  8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec  8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec  8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec  8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec  8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec  8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec  8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec  8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec  8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec  8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec  8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec  8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 501.6506ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
4.357362403s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:45:33 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.357396956s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
5.393115746s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:45:18 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 5.393146786s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................]
duration:
520.782347ms
trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:45:11 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec  8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec  8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec  8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec  8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec  8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec  8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec  8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec  8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec  8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec  8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec  8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec  8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec  8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec  8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 520.822222ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration:
404.417147ms
trace:
https://jaeger.dev.storj.io/trace/535d44beeeafe52a
FAIL (2025-12-08 17:44:45 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec  8 17:02:48.738023080 (73h34m49.565731479s) [upload=76480]: Starting upload
Dec  8 17:02:48.738095838 (73h34m49.565804247s) [upload=76480]: Done starting upload
Dec  8 17:02:48.738417931 (73h34m49.566126324s) [upload=76480]: Uploading segments...
Dec  8 17:02:48.738470600 (73h34m49.566179003s) [upload=76480]: Got next segment. Inline: false
Dec  8 17:02:48.738517579 (73h34m49.56622597s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec  8 17:02:48.738523576 (73h34m49.566231968s) [upload=76480]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec  8 17:02:48.769802661 (73h34m49.597511053s) [seg_pos={0 0}, upload=76480]: Begin upload segment...
Dec  8 17:02:48.769865494 (73h34m49.597573886s) [seg_pos={0 0}, upload=76480]: Done begin upload segment.
Dec  8 17:02:48.769907136 (73h34m49.597615538s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.769939359 (73h34m49.597647766s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770462858 (73h34m49.598171256s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770592023 (73h34m49.59830042s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770900372 (73h34m49.598608765s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.770964143 (73h34m49.598672541s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771340212 (73h34m49.599048606s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771369877 (73h34m49.599078273s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771701791 (73h34m49.599410175s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.771839972 (73h34m49.599548367s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772466849 (73h34m49.600175247s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.772890389 (73h34m49.600598785s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773397195 (73h34m49.601105589s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773517775 (73h34m49.601226156s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.773954120 (73h34m49.601662518s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.774310139 (73h34m49.602018536s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Uploading piece...
Dec  8 17:02:48.799192405 (73h34m49.626900805s) [noise=true , address=77.90.31.13:4628     , piece_id=LTI5XLVJMTZ2U373UX2L4AMJEILTDR2PYFQBAF5DKEVNQSKD5CXA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.799876711 (73h34m49.627585112s) [noise=true , address=77.90.31.12:4589     , piece_id=LJUT5EHGQZJYA2AS5S53CJULTOK4LQYV3ZDZUJAFXLIY3BPO3SFQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.807505994 (73h34m49.635214394s) [noise=true , address=77.90.31.21:4877     , piece_id=3B56C74MQDSFAXO5SAZ2BG2QMHZZWSOX2BW3DACVRYW6F2QKDMQQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec  8 17:02:48.845983839 (73h34m49.673692221s) [upload=76480]: Next returned nil segment
Dec  8 17:02:48.845997556 (73h34m49.673705948s) [upload=76480]: Waiting for error group managing segments...
Dec  8 17:02:48.853006781 (73h34m49.680715171s) [noise=true , address=145.79.227.6:4257    , piece_id=IKR5EZUXWCEVA5SQ7546KXSKWTBDYQJ7AMTVXX2T25IOOMW2TKCQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.853644007 (73h34m49.681352402s) [noise=true , address=145.79.227.12:4765   , piece_id=R6SQ4SBMCWD4DKP4FAGLPUMSOA4KRF4F2AML2TV464ZYIQ537JFA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856107946 (73h34m49.68381634s) [noise=true , address=145.79.227.22:4510   , piece_id=KVY33OABWKC4AHQXGQ2QQZBB6TNGSSSR4ZOZN56NHBGGBJUQEYLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856510829 (73h34m49.684219222s) [noise=true , address=145.79.227.26:4654   , piece_id=2FHPIZA5BL7Q4JQQZE56KLIUKCNUW7CUARG72SP4Q7YBLEV7VA7A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856772442 (73h34m49.684480837s) [noise=true , address=145.79.227.34:4907   , piece_id=YGIATRQPO6ZWFZD45CSFBSVL7L3DRNGIC5RJGPPOV6CSNXDNEDLQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.856975680 (73h34m49.684684071s) [noise=true , address=145.79.227.29:4941   , piece_id=P2Z5FPJDTTTKH3ZL4UU3JW6NC2VJ34RG4CIMAXRW6Z5EZPHUQQ6Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857242047 (73h34m49.68495045s) [noise=true , address=145.79.227.24:4583   , piece_id=VWEGXRGYJXX7H75WUTBDZFZ3HPR74VPII4K6SJPBFV62FCXS5JIQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857377588 (73h34m49.68508598s) [noise=true , address=145.79.227.28:4726   , piece_id=SBWY4VB3BN65MOVJO22NIR2HN7CS63ODABH4XH3O55NDHCCMCM2A, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857752465 (73h34m49.68546086s) [noise=true , address=145.79.227.23:4547   , piece_id=FKELFFQLGRPRX5L7FKMDZPOTBDHMKQDP6HJQUYLUDVMXOSKIMXGA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.857759093 (73h34m49.685467489s) [noise=true , address=145.79.227.23:4546   , piece_id=L2UX5LJADQOQVS7ZKHBOTHKZNXXV22W4UJ3OUGCSHWUDMHT4EYRA, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.858219382 (73h34m49.685927774s) [noise=true , address=145.79.227.27:4690   , piece_id=IWPD3QZEMPXRZVSDS7S6UEIHELJEVVWIECBE527FNIF25ZG2NP2Q, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:02:48.859176521 (73h34m49.686884917s) [noise=true , address=145.79.227.41:4086   , piece_id=QVKEINCWGAGSEBYCFWPHLGVDRLQLVT35LZCCCYF3VSKTWLY5EYRQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: <nil>
Dec  8 17:03:18.698437167 (73h35m19.526145561s) [noise=true , address=77.90.31.13:4627     , piece_id=DCNTHYBRJN5Q2IWGPKRQ7LEKYCSXOQSLQ5GI4YGPNGVQINLQKIYQ, seg_pos={0 0}, upload=76480]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF
Dec  8 17:03:18.698506535 (73h35m19.526214917s) [seg_pos={0 0}, upload=76480]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec  8 17:03:18.724186991 (73h35m19.551895387s) [upload=76480]: Done uploading segments...
Dec  8 17:03:18.724296376 (73h35m19.552004781s) [upload=76480]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 404.517792ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
8.158711403s
trace:
https://jaeger.dev.storj.io/trace/614e424f5910bbf9
FAIL (2025-12-08 17:44:28 UTC)
orbiter warning: operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 8.158725497s core uplink ap1