Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.031133786s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:08:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 2.031172839s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.930169649s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:07:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.93020165s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.921610024s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:06:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.921645438s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.942003268s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:05:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.942034332s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.961418243s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:05:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.961449065s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.961972426s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:04:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.962001911s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.121774595s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:03:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 2.121802519s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.910002334s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:02:48 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.910034472s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.050576326s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:02:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 2.050604342s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.980073415s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:01:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.980101489s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.989369693s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-17 00:00:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.989455698s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.927057441s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:59:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.927084329s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.917709144s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:58:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.917744549s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
464.482808ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:57:35 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 464.516558ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.992368456s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:57:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.992418461s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [XX..........................................................] duration:
674.367679ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:56:54 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 674.397956ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.231905116s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:56:41 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.232035397s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.998477919s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:56:35 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.998510775s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.117700503s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:56:19 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.117728801s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.91479768s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:56:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.914837612s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [XXX.........................................................] duration:
413.521179ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:55:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 413.572022ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.254559355s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:55:42 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.254597671s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.881066767s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:55:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.881095238s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
269.661386ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:55:40 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 269.692275ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
908.682219ms trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:55:23 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 908.710825ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
678.065162ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:55:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 678.086127ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.729704591s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:55:13 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.729721921s
core
uplink
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.020737988s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:55:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 2.020767908s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.290242584s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:55:08 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.290270828s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.6592034s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:55:05 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.659519808s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [XXXX........................................................] duration:
924.34497ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:55:05 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 924.373537ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
258.512696ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:55:02 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 258.563222ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
998.812458ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:54:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 998.843183ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.727014583s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:54:45 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.727044767s
core
uplink
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.954561803s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:54:40 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.954648761s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.183565317s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:54:33 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.183599876s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
239.627021ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:54:31 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 239.665575ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.179697671s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:54:22 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.17975794s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.320427932s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:54:20 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.320519767s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [XXX.........................................................] duration:
603.128073ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:54:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 603.155973ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.197019647s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:54:00 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.197049086s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
4.317117989s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:53:59 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.317204688s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [XXXX........................................................] duration:
556.5803ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:53:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 556.606784ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [XXXXX.......................................................] duration:
572.960221ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:53:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 572.995097ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.249246478s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:53:56 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.249277102s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
248.665075ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:53:54 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 248.694701ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.850275578s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:53:45 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.850306732s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.131987304s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:53:40 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.132100013s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.173647804s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:53:35 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.173674476s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.933862188s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:53:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.933906346s
core
uplink
uk-select-1
uk1-placement-17
|