Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.689680491s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:51:46 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.689707682s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..XXX.......................................................] duration:
4.254624127s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:51:41 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.254650921s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [...XXXXX....................................................] duration:
594.574796ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:51:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 594.632343ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.928856836s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:51:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.928886319s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
255.71424ms trace:
https://jaeger.dev.storj.io/trace/7d774feb475c25d5
FAIL (2026-03-16 23:51:22 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 255.801627ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.201544575s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:51:18 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.201574262s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
719.654214ms target:
https://ap1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/c9cd61a7d6477ab
FAIL (2026-03-16 23:51:16 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.146.6:443: i/o timeout
Duration: 719.692498ms
core
satellite
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [....XXXXX...................................................] duration:
585.089848ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:51:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 585.13274ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
279.886525ms trace:
https://jaeger.dev.storj.io/trace/18df21c1c2c0ed9a
FAIL (2026-03-16 23:51:05 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 279.921772ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
216.265659ms trace:
https://jaeger.dev.storj.io/trace/7d774feb475c25d5
FAIL (2026-03-16 23:50:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 216.290654ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
264.772517ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:50:47 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 264.804765ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
889.829779ms trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:50:42 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 889.856499ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.007969259s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:50:41 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.008055987s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [.XXXXXXXX...................................................] duration:
502.121421ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:50:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 502.146362ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.....XXXXX..................................................] duration:
547.474493ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:50:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 547.515809ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
719.483593ms target:
https://ap1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/c9cd61a7d6477ab
FAIL (2026-03-16 23:50:32 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.146.6:443: i/o timeout
Duration: 719.511321ms
core
satellite
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...XXX......................................................] duration:
4.596564949s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:50:31 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.596600183s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.057335528s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:50:28 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.057441359s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.994370404s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:50:26 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.994396794s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.01385637s trace:
https://jaeger.dev.storj.io/trace/18e92533cb874ee7
FAIL (2026-03-16 23:50:25 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.013886048s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....XXX.....................................................] duration:
4.384271396s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:50:11 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.384316139s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [......XXXXX.................................................] duration:
751.666724ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:50:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 751.747819ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
216.055746ms trace:
https://jaeger.dev.storj.io/trace/7d774feb475c25d5
FAIL (2026-03-16 23:50:08 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 216.086718ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [..XXXXXXXX..................................................] duration:
752.745072ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:50:07 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 752.772417ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
245.369428ms trace:
https://jaeger.dev.storj.io/trace/18df21c1c2c0ed9a
FAIL (2026-03-16 23:49:56 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 245.395155ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
251.163395ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:49:53 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 251.203739ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.107176521s trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:49:53 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.10725295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
2.031870977s trace:
https://jaeger.dev.storj.io/trace/18e92533cb874ee7
FAIL (2026-03-16 23:49:47 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.031906539s
core
uplink
ap1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
725.672912ms target:
https://ap1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/c9cd61a7d6477ab
FAIL (2026-03-16 23:49:40 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.146.6:443: i/o timeout
Duration: 725.69909ms
core
satellite
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......XXXXX................................................] duration:
810.258545ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:49:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 810.311481ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration:
4.348803308s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:49:33 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.348886167s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [...XXXXXXXX.................................................] duration:
547.084749ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:49:29 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 547.112655ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.264542893s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:49:28 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.264640281s
core
uplink
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.863014732s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:49:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.863051908s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
230.999738ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:49:27 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 231.088763ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
264.096796ms trace:
https://jaeger.dev.storj.io/trace/7d774feb475c25d5
FAIL (2026-03-16 23:49:18 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 264.126043ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
252.220406ms trace:
https://jaeger.dev.storj.io/trace/18df21c1c2c0ed9a
FAIL (2026-03-16 23:49:18 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 252.246539ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.178046975s trace:
https://jaeger.dev.storj.io/trace/71a8d4082dd7d62d
FAIL (2026-03-16 23:49:17 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.178089012s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
3 out of 60 is failed (threshold=3) [XXX.........................................................] duration:
2.032602088s trace:
https://jaeger.dev.storj.io/trace/18e92533cb874ee7
FAIL (2026-03-16 23:49:04 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.032631217s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
919.462422ms trace:
https://jaeger.dev.storj.io/trace/6fdbf3c7687a0ce5
FAIL (2026-03-16 23:49:01 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 919.495807ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.remote)antiblip:
3 out of 60 is failed (threshold=3) [......XXX...................................................] duration:
4.304076802s trace:
https://jaeger.dev.storj.io/trace/488c16a528e2a255
FAIL (2026-03-16 23:48:57 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.304109021s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.826285438s trace:
https://jaeger.dev.storj.io/trace/6f888b1e7fd43d
FAIL (2026-03-16 23:48:52 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.82631895s
core
uplink
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.965413162s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:48:43 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.965443641s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [........XXXXX...............................................] duration:
615.001726ms trace:
https://jaeger.dev.storj.io/trace/22ca2a71d8e0411f
FAIL (2026-03-16 23:48:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 615.050549ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
233.044936ms trace:
https://jaeger.dev.storj.io/trace/18df21c1c2c0ed9a
FAIL (2026-03-16 23:48:40 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 233.075383ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
713.324874ms target:
https://ap1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/c9cd61a7d6477ab
FAIL (2026-03-16 23:48:38 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.146.6:443: i/o timeout
Duration: 713.44458ms
core
satellite
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
243.761757ms trace:
https://jaeger.dev.storj.io/trace/5ee26ff87c6300e4
FAIL (2026-03-16 23:48:36 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 243.863365ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [....XXXXXXXX................................................] duration:
565.26187ms trace:
https://jaeger.dev.storj.io/trace/70403794f26a4156
FAIL (2026-03-16 23:48:29 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 565.291497ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.992273625s trace:
https://jaeger.dev.storj.io/trace/1f4071450dd807fc
FAIL (2026-03-16 23:48:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 ======LOGS====== Mar 16 23:43:18.026386127 (252h43m15.623157157s) [upload=283106]: Starting upload Mar 16 23:43:18.026569032 (252h43m15.623340062s) [upload=283106]: Done starting upload Mar 16 23:43:18.026683049 (252h43m15.623454058s) [upload=283106]: Uploading segments... Mar 16 23:43:18.027247579 (252h43m15.62401861s) [upload=283106]: Got next segment. Inline: false Mar 16 23:43:18.027369023 (252h43m15.624140057s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 16 23:43:18.027377197 (252h43m15.624148225s) [upload=283106]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Mar 16 23:43:18.129722287 (252h43m15.726493311s) [upload=283106]: Next segment err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0 Mar 16 23:43:18.129747413 (252h43m15.726518446s) [upload=283106]: Done uploading segments... Mar 16 23:43:18.129809790 (252h43m15.726580818s) [upload=283106]: Upload finished. err: metaclient: not enough nodes: not enough nodes: requested from cache 20, found 0
Duration: 1.992295988s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
329.402261ms trace:
https://jaeger.dev.storj.io/trace/7d774feb475c25d5
FAIL (2026-03-16 23:48:26 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 329.436383ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|