Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 415.531436ms FAIL (2025-07-24 10:42:42 UTC) orbiter warning: uplink: too many requests Duration: 415.564969ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 414.05308ms FAIL (2025-07-24 10:41:31 UTC) orbiter warning: uplink: too many requests Duration: 414.083777ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 457.880667ms FAIL (2025-07-24 10:41:04 UTC) orbiter warning: uplink: too many requests Duration: 457.929369ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 562.379607ms FAIL (2025-07-24 10:40:08 UTC) orbiter warning: uplink: too many requests Duration: 562.418082ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 436.824035ms FAIL (2025-07-24 10:39:51 UTC) orbiter warning: uplink: too many requests Duration: 436.849087ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 70.103238ms FAIL (2025-07-24 10:38:46 UTC) orbiter warning: uplink: too many requests Duration: 70.121985ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 32.643972ms FAIL (2025-07-24 10:21:29 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 32.759034ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.144758ms FAIL (2025-07-24 10:20:54 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 30.163718ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 40.63596ms FAIL (2025-07-24 10:20:00 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 40.666312ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 36.654221ms FAIL (2025-07-24 10:19:39 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 36.679845ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 54.680172ms FAIL (2025-07-24 10:19:16 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 54.709338ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 33.401631ms FAIL (2025-07-24 10:18:13 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 33.430541ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 27.456291ms FAIL (2025-07-24 10:17:41 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 27.501369ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 25.78404ms FAIL (2025-07-24 10:17:17 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 25.81278ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.088843ms FAIL (2025-07-24 10:16:33 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 30.110199ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 29.384563ms FAIL (2025-07-24 10:15:56 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 29.441712ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 28.43574ms FAIL (2025-07-24 10:14:58 UTC) uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 28.482919ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 27.977544ms FAIL (2025-07-24 10:13:49 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 28.008118ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 27.064908ms FAIL (2025-07-24 10:13:27 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 27.154053ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 298.9798ms FAIL (2025-07-24 07:45:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 299.007077ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 281.326697ms FAIL (2025-07-24 07:44:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 281.361507ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 291.804252ms FAIL (2025-07-24 07:44:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 291.845135ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 334.098578ms FAIL (2025-07-24 07:43:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 334.128767ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 489.195122ms FAIL (2025-07-24 07:42:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 489.231639ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 278.375561ms FAIL (2025-07-24 07:41:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 278.500485ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 298.927831ms FAIL (2025-07-24 07:41:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 298.990656ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 342.635619ms FAIL (2025-07-24 07:40:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 342.659269ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 539.192319ms FAIL (2025-07-24 07:39:24 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 539.234847ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 555.310654ms FAIL (2025-07-24 07:38:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 555.338016ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 322.558274ms FAIL (2025-07-24 07:38:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 322.605184ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 611.467164ms FAIL (2025-07-24 07:37:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 611.498483ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 415.766366ms FAIL (2025-07-24 07:36:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 415.796798ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 276.264215ms FAIL (2025-07-24 07:36:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 276.458346ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 544.27737ms FAIL (2025-07-24 07:35:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 544.304003ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 244.744802ms FAIL (2025-07-24 07:34:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 244.864193ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 231.703051ms FAIL (2025-07-24 07:33:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 231.742961ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 234.414772ms FAIL (2025-07-24 07:32:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 234.446938ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 241.204326ms FAIL (2025-07-24 07:31:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 241.24692ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 333.469149ms FAIL (2025-07-24 07:30:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 333.526558ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 282.388016ms FAIL (2025-07-24 07:29:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 282.412956ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 238.663624ms FAIL (2025-07-24 07:28:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 238.692299ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 241.858301ms FAIL (2025-07-24 07:27:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 241.885256ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 283.737759ms FAIL (2025-07-24 07:27:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 283.768884ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 231.228119ms FAIL (2025-07-24 07:26:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 231.266348ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 281.526928ms FAIL (2025-07-24 07:26:24 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 281.556672ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 281.134935ms FAIL (2025-07-24 07:25:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 281.16813ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 260.860053ms FAIL (2025-07-24 07:25:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 260.892184ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 237.304996ms FAIL (2025-07-24 07:24:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 237.336144ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 288.296006ms FAIL (2025-07-24 07:23:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 288.320487ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 357.627327ms FAIL (2025-07-24 07:23:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 357.649069ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)