Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 223.360192ms FAIL (2025-07-24 07:22:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 223.394258ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 290.58765ms FAIL (2025-07-24 07:21:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 290.61895ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 247.972978ms FAIL (2025-07-24 07:21:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 248.006107ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 267.75417ms FAIL (2025-07-24 07:20:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 267.780092ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 262.220848ms FAIL (2025-07-24 07:20:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 262.266939ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 250.28859ms FAIL (2025-07-24 07:19:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 250.3465ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 450.048833ms FAIL (2025-07-24 07:17:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 450.081383ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 282.049448ms FAIL (2025-07-24 07:16:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 282.077894ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 335.471216ms FAIL (2025-07-24 07:16:05 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 335.498048ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 228.776961ms FAIL (2025-07-24 07:15:39 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 228.808095ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 236.158623ms FAIL (2025-07-24 07:15:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 236.187059ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 256.945009ms FAIL (2025-07-24 07:14:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 256.969688ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 252.725039ms FAIL (2025-07-24 07:13:51 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 252.751878ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 569.778976ms FAIL (2025-07-24 07:13:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 569.807179ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 272.207142ms FAIL (2025-07-24 07:12:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 272.231948ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 231.299623ms FAIL (2025-07-24 07:11:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 231.39587ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 265.124436ms FAIL (2025-07-24 07:10:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 265.146186ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 244.369573ms FAIL (2025-07-24 07:09:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 244.412602ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 249.15166ms FAIL (2025-07-24 07:09:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 249.258311ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 253.857228ms FAIL (2025-07-24 07:09:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 253.890468ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 248.117196ms FAIL (2025-07-24 07:08:14 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 248.151935ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 300.562114ms FAIL (2025-07-24 07:07:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 300.590622ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 308.937894ms FAIL (2025-07-24 07:06:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 308.968085ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 268.120458ms FAIL (2025-07-24 07:06:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 268.158227ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 349.662266ms FAIL (2025-07-24 07:04:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 349.687878ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 314.181039ms FAIL (2025-07-24 07:04:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 314.214267ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 327.803914ms FAIL (2025-07-24 07:03:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 327.838308ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 711.34227ms FAIL (2025-07-24 07:02:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 711.397422ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00130442s FAIL (2025-07-24 07:01:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== : <nil> Jul 24 06:52:39.461381632 (128h2m21.309988704s) [upload=102366]: Starting upload Jul 24 06:52:39.461468856 (128h2m21.310075936s) [upload=102366]: Done starting upload Jul 24 06:52:39.461583692 (128h2m21.310190772s) [upload=102366]: Uploading segments... Jul 24 06:52:39.461630385 (128h2m21.310237473s) [upload=102366]: Got next segment. Inline: true Jul 24 06:52:39.461636994 (128h2m21.310244074s) [upload=102366]: Waiting for error group managing segments... Jul 24 06:52:39.461702719 (128h2m21.310309798s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:52:39.461708256 (128h2m21.310315336s) [upload=102366]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:52:39.461711416 (128h2m21.310318496s) [upload=102366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:52:39.537381735 (128h2m21.385988816s) [upload=102366]: Done uploading segments... Jul 24 06:52:39.537404300 (128h2m21.386011426s) [upload=102366]: Upload finished. err: <nil> Jul 24 06:53:29.309127659 (128h3m11.157734748s) [upload=102383]: Starting upload Jul 24 06:53:29.309250454 (128h3m11.157857537s) [upload=102383]: Done starting upload Jul 24 06:53:29.309367817 (128h3m11.1579749s) [upload=102383]: Uploading segments... Jul 24 06:53:29.309453537 (128h3m11.158060626s) [upload=102383]: Got next segment. Inline: true Jul 24 06:53:29.309460501 (128h3m11.158067586s) [upload=102383]: Waiting for error group managing segments... Jul 24 06:53:29.309513712 (128h3m11.158120792s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:53:29.309520401 (128h3m11.158127484s) [upload=102383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:53:29.309523633 (128h3m11.158130713s) [upload=102383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:53:29.384857508 (128h3m11.233464595s) [upload=102383]: Done uploading segments... Jul 24 06:53:29.384895729 (128h3m11.233502809s) [upload=102383]: Upload finished. err: <nil> Jul 24 06:54:37.429619610 (128h4m19.278226681s) [upload=102396]: Starting upload Jul 24 06:54:37.429698591 (128h4m19.278305684s) [upload=102396]: Done starting upload Jul 24 06:54:37.429805514 (128h4m19.278412596s) [upload=102396]: Uploading segments... Jul 24 06:54:37.429880628 (128h4m19.278487719s) [upload=102396]: Got next segment. Inline: true Jul 24 06:54:37.429887393 (128h4m19.278494484s) [upload=102396]: Waiting for error group managing segments... Jul 24 06:54:37.429955363 (128h4m19.278562448s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:54:37.429961460 (128h4m19.278568545s) [upload=102396]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:54:37.429964897 (128h4m19.278571983s) [upload=102396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:54:37.685771590 (128h4m19.534378676s) [upload=102396]: Done uploading segments... Jul 24 06:54:37.685796826 (128h4m19.534403907s) [upload=102396]: Upload finished. err: <nil> Jul 24 06:55:31.115463464 (128h5m12.964070546s) [upload=102407]: Starting upload Jul 24 06:55:31.115526451 (128h5m12.964133521s) [upload=102407]: Done starting upload Jul 24 06:55:31.115594591 (128h5m12.964201663s) [upload=102407]: Uploading segments... Jul 24 06:55:31.115635026 (128h5m12.964242114s) [upload=102407]: Got next segment. Inline: true Jul 24 06:55:31.115640361 (128h5m12.96424744s) [upload=102407]: Waiting for error group managing segments... Jul 24 06:55:31.115683294 (128h5m12.964290373s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:55:31.115688944 (128h5m12.964296023s) [upload=102407]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:55:31.115692335 (128h5m12.964299415s) [upload=102407]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:55:31.192800787 (128h5m13.041407872s) [upload=102407]: Done uploading segments... Jul 24 06:55:31.192824693 (128h5m13.041431774s) [upload=102407]: Upload finished. err: <nil> Jul 24 06:56:18.752843554 (128h6m0.601450628s) [upload=102420]: Starting upload Jul 24 06:56:18.752924636 (128h6m0.60153173s) [upload=102420]: Done starting upload Jul 24 06:56:18.753001742 (128h6m0.601608808s) [upload=102420]: Uploading segments... Jul 24 06:56:18.753040402 (128h6m0.601647488s) [upload=102420]: Got next segment. Inline: true Jul 24 06:56:18.753045016 (128h6m0.601652094s) [upload=102420]: Waiting for error group managing segments... Jul 24 06:56:18.753081096 (128h6m0.601688161s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:56:18.753085252 (128h6m0.60169233s) [upload=102420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:56:18.753088486 (128h6m0.601695569s) [upload=102420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:56:18.824653503 (128h6m0.673260572s) [upload=102420]: Done uploading segments... Jul 24 06:56:18.824672083 (128h6m0.673279152s) [upload=102420]: Upload finished. err: <nil> Jul 24 06:57:31.128809188 (128h7m12.977416269s) [upload=102437]: Starting upload Jul 24 06:57:31.128939331 (128h7m12.977546412s) [upload=102437]: Done starting upload Jul 24 06:57:31.129033596 (128h7m12.977640678s) [upload=102437]: Uploading segments... Jul 24 06:57:31.129226353 (128h7m12.977833444s) [upload=102437]: Got next segment. Inline: true Jul 24 06:57:31.129233724 (128h7m12.977840805s) [upload=102437]: Waiting for error group managing segments... Jul 24 06:57:31.129332637 (128h7m12.977939718s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:57:31.129339636 (128h7m12.97794672s) [upload=102437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:57:31.129343704 (128h7m12.977950787s) [upload=102437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:57:31.198118364 (128h7m13.046725454s) [upload=102437]: Done uploading segments... Jul 24 06:57:31.198139624 (128h7m13.046746704s) [upload=102437]: Upload finished. err: <nil> Jul 24 06:58:03.382672004 (128h7m45.231279087s) [upload=102443]: Starting upload Jul 24 06:58:03.382743242 (128h7m45.231350333s) [upload=102443]: Done starting upload Jul 24 06:58:03.382834244 (128h7m45.231441318s) [upload=102443]: Uploading segments... Jul 24 06:58:03.382882835 (128h7m45.231489926s) [upload=102443]: Got next segment. Inline: true Jul 24 06:58:03.382890668 (128h7m45.23149776s) [upload=102443]: Waiting for error group managing segments... Jul 24 06:58:03.382944235 (128h7m45.231551318s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:58:03.382951284 (128h7m45.231558367s) [upload=102443]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:58:03.382956398 (128h7m45.231563482s) [upload=102443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:58:03.480957194 (128h7m45.329564284s) [upload=102443]: Done uploading segments... Jul 24 06:58:03.480981301 (128h7m45.329588385s) [upload=102443]: Upload finished. err: <nil> Jul 24 06:59:15.844046733 (128h8m57.69265381s) [upload=102460]: Starting upload Jul 24 06:59:15.844121300 (128h8m57.692728384s) [upload=102460]: Done starting upload Jul 24 06:59:15.844235298 (128h8m57.692842367s) [upload=102460]: Uploading segments... Jul 24 06:59:15.844296452 (128h8m57.692903535s) [upload=102460]: Got next segment. Inline: true Jul 24 06:59:15.844302862 (128h8m57.692909932s) [upload=102460]: Waiting for error group managing segments... Jul 24 06:59:15.844347805 (128h8m57.692954877s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 06:59:15.844353609 (128h8m57.692960679s) [upload=102460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 06:59:15.844372778 (128h8m57.692979858s) [upload=102460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 06:59:15.930005058 (128h8m57.778612145s) [upload=102460]: Done uploading segments... Jul 24 06:59:15.930028311 (128h8m57.778635398s) [upload=102460]: Upload finished. err: <nil> Jul 24 07:00:25.495256680 (128h10m7.343863784s) [upload=102474]: Starting upload Jul 24 07:00:25.495355857 (128h10m7.343962928s) [upload=102474]: Done starting upload Jul 24 07:00:25.495459398 (128h10m7.344066467s) [upload=102474]: Uploading segments... Jul 24 07:00:25.495498119 (128h10m7.344105205s) [upload=102474]: Got next segment. Inline: true Jul 24 07:00:25.495503117 (128h10m7.344110196s) [upload=102474]: Waiting for error group managing segments... Jul 24 07:00:25.495556639 (128h10m7.344163709s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:25.495563817 (128h10m7.344170898s) [upload=102474]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:25.495567185 (128h10m7.344174265s) [upload=102474]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:25.677278695 (128h10m7.525885782s) [upload=102474]: Done uploading segments... Jul 24 07:00:25.677301712 (128h10m7.525908792s) [upload=102474]: Upload finished. err: <nil> Jul 24 07:00:49.806969474 (128h10m31.655576557s) [upload=102478]: Starting upload Jul 24 07:00:49.807081662 (128h10m31.655688742s) [upload=102478]: Done starting upload Jul 24 07:00:49.807198486 (128h10m31.655805565s) [upload=102478]: Uploading segments... Jul 24 07:00:49.807248321 (128h10m31.655855406s) [upload=102478]: Got next segment. Inline: true Jul 24 07:00:49.807254838 (128h10m31.655861917s) [upload=102478]: Waiting for error group managing segments... Jul 24 07:00:49.807311812 (128h10m31.655918892s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 24 07:00:49.807317014 (128h10m31.655924094s) [upload=102478]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 24 07:00:49.807320337 (128h10m31.655927416s) [upload=102478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 24 07:00:49.922154053 (128h10m31.770761136s) [upload=102478]: Done uploading segments... Jul 24 07:00:49.922176880 (128h10m31.770783964s) [upload=102478]: Upload finished. err: <nil> Duration: 30.001339208s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.669832902s FAIL (2025-07-23 21:58:08 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.669868656s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.736855658s FAIL (2025-07-23 21:57:27 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.736886134s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 948.941594ms FAIL (2025-07-23 21:57:03 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 948.975963ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.977081882s FAIL (2025-07-23 21:56:50 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.977113473s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 866.940682ms FAIL (2025-07-23 21:56:21 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 866.973598ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.769865571s FAIL (2025-07-23 21:56:13 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.769893996s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.724751778s FAIL (2025-07-23 21:55:47 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.724784231s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 826.677703ms FAIL (2025-07-23 21:55:40 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 826.705786ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.726278301s FAIL (2025-07-23 21:54:42 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.726308409s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 957.177062ms FAIL (2025-07-23 21:54:27 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 957.198566ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.798948278s FAIL (2025-07-23 21:53:54 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.798975315s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.871747592s FAIL (2025-07-23 21:53:35 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.871776293s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 959.255902ms FAIL (2025-07-23 21:53:25 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 959.290181ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 972.358162ms FAIL (2025-07-23 21:52:51 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 972.385497ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.893076453s FAIL (2025-07-23 21:52:46 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.893166852s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 888.405263ms FAIL (2025-07-23 21:52:09 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 888.437859ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 920.977651ms FAIL (2025-07-23 21:51:53 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 921.004181ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.755648264s FAIL (2025-07-23 21:51:45 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.755675132s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 3.032827685s FAIL (2025-07-23 21:51:13 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 3.032856678s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 907.178124ms FAIL (2025-07-23 21:50:48 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 907.210292ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.803195906s FAIL (2025-07-23 21:50:43 UTC) orbiter warning: uplink: metainfo: metaclient: context deadline exceeded Duration: 2.803232058s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)