Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 523.077378ms FAIL (2025-07-07 21:12:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 523.587082ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 247.302728ms FAIL (2025-07-07 21:12:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 247.352478ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 253.845028ms FAIL (2025-07-07 21:11:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 253.873503ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 494.763207ms FAIL (2025-07-07 21:11:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 494.787816ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 596.295976ms FAIL (2025-07-07 21:10:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 596.324616ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 245.514465ms FAIL (2025-07-07 21:10:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 245.543605ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 233.936679ms FAIL (2025-07-07 21:09:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 233.96907ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 571.425943ms FAIL (2025-07-07 21:09:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 571.451451ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 555.794807ms FAIL (2025-07-07 21:08:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 555.815104ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 227.007192ms FAIL (2025-07-07 21:08:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 227.039744ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 588.85411ms FAIL (2025-07-07 21:08:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 588.982616ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 272.615802ms FAIL (2025-07-07 21:07:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 272.646084ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 527.78078ms FAIL (2025-07-07 21:07:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 527.820552ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 270.535006ms FAIL (2025-07-07 21:06:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 270.567337ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 598.989335ms FAIL (2025-07-07 21:06:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 599.028865ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 245.303168ms FAIL (2025-07-07 21:05:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 245.344304ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 603.245457ms FAIL (2025-07-07 21:05:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 603.719291ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 245.247173ms FAIL (2025-07-07 21:04:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 245.330126ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 525.708332ms FAIL (2025-07-07 21:04:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 525.728977ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 503.726033ms FAIL (2025-07-07 21:03:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 503.757979ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 330.049368ms FAIL (2025-07-07 21:03:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 330.114708ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 241.526394ms FAIL (2025-07-07 21:02:59 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 241.555298ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 558.796778ms FAIL (2025-07-07 21:02:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 558.842174ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 230.21668ms FAIL (2025-07-07 21:02:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 230.30781ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 648.153944ms FAIL (2025-07-07 21:01:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 648.217762ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 262.478726ms FAIL (2025-07-07 21:01:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 262.498785ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 237.253968ms FAIL (2025-07-07 21:01:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 237.348819ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 682.473066ms FAIL (2025-07-07 21:00:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 682.494521ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 246.946005ms FAIL (2025-07-07 21:00:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 246.973282ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 236.613115ms FAIL (2025-07-07 20:59:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 236.638873ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 588.976705ms FAIL (2025-07-07 20:59:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 589.007633ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 397.634716ms FAIL (2025-07-07 20:59:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 397.67244ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 655.347978ms FAIL (2025-07-07 20:59:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 655.411441ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 431.660948ms FAIL (2025-07-07 20:58:51 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 431.699209ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 483.832418ms FAIL (2025-07-07 20:58:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 483.852014ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 525.398688ms FAIL (2025-07-07 20:58:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 525.424621ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 223.557362ms FAIL (2025-07-07 20:57:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 223.605765ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 561.621661ms FAIL (2025-07-07 20:57:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 561.642221ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 279.838418ms FAIL (2025-07-07 20:56:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 279.863051ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 587.674935ms FAIL (2025-07-07 20:56:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 587.726163ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 256.654363ms FAIL (2025-07-07 20:56:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 256.689385ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 845.474194ms FAIL (2025-07-07 20:55:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 845.498285ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 261.446296ms FAIL (2025-07-07 20:55:05 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 261.47862ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 662.999886ms FAIL (2025-07-07 20:54:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 663.048466ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 539.059729ms FAIL (2025-07-07 20:53:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 539.088784ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 270.960805ms FAIL (2025-07-07 20:53:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 270.988447ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 625.820304ms FAIL (2025-07-07 20:53:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 625.846814ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 498.336033ms FAIL (2025-07-07 20:53:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 498.367794ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 560.723962ms FAIL (2025-07-07 20:52:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 560.750172ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 248.259094ms FAIL (2025-07-07 20:52:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ng upload Jul 7 20:27:39.636855363 (347h16m6.563150052s) [upload=278131]: Done starting upload Jul 7 20:27:39.636922474 (347h16m6.563217146s) [upload=278131]: Uploading segments... Jul 7 20:27:39.636956546 (347h16m6.563251241s) [upload=278131]: Got next segment. Inline: true Jul 7 20:27:39.636960880 (347h16m6.563255567s) [upload=278131]: Waiting for error group managing segments... Jul 7 20:27:39.636996117 (347h16m6.56329079s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:27:39.637000332 (347h16m6.563295019s) [upload=278131]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:27:39.637002887 (347h16m6.563297559s) [upload=278131]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:27:39.722613186 (347h16m6.648907874s) [upload=278131]: Done uploading segments... Jul 7 20:27:39.722632626 (347h16m6.648927425s) [upload=278131]: Upload finished. err: <nil> Jul 7 20:28:18.207197318 (347h16m45.133491995s) [upload=278139]: Starting upload Jul 7 20:28:18.207272806 (347h16m45.133567492s) [upload=278139]: Done starting upload Jul 7 20:28:18.207341376 (347h16m45.133636063s) [upload=278139]: Uploading segments... Jul 7 20:28:18.207378374 (347h16m45.13367307s) [upload=278139]: Got next segment. Inline: true Jul 7 20:28:18.207394885 (347h16m45.133689572s) [upload=278139]: Waiting for error group managing segments... Jul 7 20:28:18.207450525 (347h16m45.133745211s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:28:18.207456627 (347h16m45.133751313s) [upload=278139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:28:18.207542805 (347h16m45.133837491s) [upload=278139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:28:18.284154465 (347h16m45.210449156s) [upload=278139]: Done uploading segments... Jul 7 20:28:18.284177436 (347h16m45.210472123s) [upload=278139]: Upload finished. err: <nil> Jul 7 20:29:00.049882670 (347h17m26.976177342s) [upload=278148]: Starting upload Jul 7 20:29:00.049949347 (347h17m26.976244042s) [upload=278148]: Done starting upload Jul 7 20:29:00.050028716 (347h17m26.976323392s) [upload=278148]: Uploading segments... Jul 7 20:29:00.050071742 (347h17m26.976366437s) [upload=278148]: Got next segment. Inline: true Jul 7 20:29:00.050077286 (347h17m26.97637198s) [upload=278148]: Waiting for error group managing segments... Jul 7 20:29:00.050136779 (347h17m26.976431476s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:00.050152728 (347h17m26.976447424s) [upload=278148]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:00.050158614 (347h17m26.976453307s) [upload=278148]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:00.289880244 (347h17m27.216174931s) [upload=278148]: Done uploading segments... Jul 7 20:29:00.289905877 (347h17m27.216200569s) [upload=278148]: Upload finished. err: <nil> Jul 7 20:29:16.108243601 (347h17m43.034538294s) [upload=278152]: Starting upload Jul 7 20:29:16.108348286 (347h17m43.034642973s) [upload=278152]: Done starting upload Jul 7 20:29:16.108440221 (347h17m43.034734912s) [upload=278152]: Uploading segments... Jul 7 20:29:16.108541904 (347h17m43.034836605s) [upload=278152]: Got next segment. Inline: true Jul 7 20:29:16.108554425 (347h17m43.034849122s) [upload=278152]: Waiting for error group managing segments... Jul 7 20:29:16.108613732 (347h17m43.034908418s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:29:16.108622584 (347h17m43.034917283s) [upload=278152]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:29:16.108627777 (347h17m43.034922521s) [upload=278152]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:29:16.177692122 (347h17m43.103986812s) [upload=278152]: Done uploading segments... Jul 7 20:29:16.177715711 (347h17m43.104010398s) [upload=278152]: Upload finished. err: <nil> Jul 7 20:30:07.200047120 (347h18m34.126341815s) [upload=278164]: Starting upload Jul 7 20:30:07.200122665 (347h18m34.126417361s) [upload=278164]: Done starting upload Jul 7 20:30:07.200215970 (347h18m34.126510646s) [upload=278164]: Uploading segments... Jul 7 20:30:07.200264325 (347h18m34.126559021s) [upload=278164]: Got next segment. Inline: true Jul 7 20:30:07.200270003 (347h18m34.126564697s) [upload=278164]: Waiting for error group managing segments... Jul 7 20:30:07.200316012 (347h18m34.126610706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:30:07.200322371 (347h18m34.12661706s) [upload=278164]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:30:07.200326719 (347h18m34.126621393s) [upload=278164]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:30:07.271934580 (347h18m34.198229277s) [upload=278164]: Done uploading segments... Jul 7 20:30:07.271992347 (347h18m34.198287045s) [upload=278164]: Upload finished. err: <nil> Jul 7 20:31:07.342724770 (347h19m34.269019468s) [upload=278177]: Starting upload Jul 7 20:31:07.342823078 (347h19m34.269117776s) [upload=278177]: Done starting upload Jul 7 20:31:07.342930361 (347h19m34.269225041s) [upload=278177]: Uploading segments... Jul 7 20:31:07.342999457 (347h19m34.269294153s) [upload=278177]: Got next segment. Inline: true Jul 7 20:31:07.343006337 (347h19m34.269301028s) [upload=278177]: Waiting for error group managing segments... Jul 7 20:31:07.343102972 (347h19m34.269397662s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:07.343108760 (347h19m34.269403447s) [upload=278177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:07.343114220 (347h19m34.269408907s) [upload=278177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:07.405128288 (347h19m34.331422974s) [upload=278177]: Done uploading segments... Jul 7 20:31:07.405148074 (347h19m34.331442761s) [upload=278177]: Upload finished. err: <nil> Jul 7 20:31:22.595191817 (347h19m49.521486506s) [upload=278182]: Starting upload Jul 7 20:31:22.595271255 (347h19m49.521565951s) [upload=278182]: Done starting upload Jul 7 20:31:22.595389433 (347h19m49.521684106s) [upload=278182]: Uploading segments... Jul 7 20:31:22.595486207 (347h19m49.521780903s) [upload=278182]: Got next segment. Inline: true Jul 7 20:31:22.595495146 (347h19m49.521789839s) [upload=278182]: Waiting for error group managing segments... Jul 7 20:31:22.595598902 (347h19m49.52189359s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:31:22.595607776 (347h19m49.521902471s) [upload=278182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:31:22.595611583 (347h19m49.521906258s) [upload=278182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:31:22.668890007 (347h19m49.595184696s) [upload=278182]: Done uploading segments... Jul 7 20:31:22.668910303 (347h19m49.595204991s) [upload=278182]: Upload finished. err: <nil> Jul 7 20:32:09.078329483 (347h20m36.00462417s) [upload=278191]: Starting upload Jul 7 20:32:09.078386523 (347h20m36.004681211s) [upload=278191]: Done starting upload Jul 7 20:32:09.078497920 (347h20m36.004792593s) [upload=278191]: Uploading segments... Jul 7 20:32:09.078571788 (347h20m36.004866477s) [upload=278191]: Got next segment. Inline: true Jul 7 20:32:09.078577217 (347h20m36.004871903s) [upload=278191]: Waiting for error group managing segments... Jul 7 20:32:09.078633608 (347h20m36.004928283s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:09.078638921 (347h20m36.004933597s) [upload=278191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:09.078642500 (347h20m36.004937176s) [upload=278191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:09.155298109 (347h20m36.08159281s) [upload=278191]: Done uploading segments... Jul 7 20:32:09.155320753 (347h20m36.081615446s) [upload=278191]: Upload finished. err: <nil> Jul 7 20:32:48.912216516 (347h21m15.838511191s) [upload=278200]: Starting upload Jul 7 20:32:48.912320542 (347h21m15.838615218s) [upload=278200]: Done starting upload Jul 7 20:32:48.912413059 (347h21m15.838707755s) [upload=278200]: Uploading segments... Jul 7 20:32:48.912536235 (347h21m15.838830927s) [upload=278200]: Got next segment. Inline: true Jul 7 20:32:48.912542437 (347h21m15.838837129s) [upload=278200]: Waiting for error group managing segments... Jul 7 20:32:48.912602991 (347h21m15.83889767s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:32:48.912611416 (347h21m15.83890612s) [upload=278200]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:32:48.912617199 (347h21m15.838911904s) [upload=278200]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:32:48.970632380 (347h21m15.896927057s) [upload=278200]: Done uploading segments... Jul 7 20:32:48.970653336 (347h21m15.896948133s) [upload=278200]: Upload finished. err: <nil> Jul 7 20:33:36.422935558 (347h22m3.349230234s) [upload=278214]: Starting upload Jul 7 20:33:36.423043066 (347h22m3.349337752s) [upload=278214]: Done starting upload Jul 7 20:33:36.423110686 (347h22m3.349405359s) [upload=278214]: Uploading segments... Jul 7 20:33:36.423151673 (347h22m3.349446415s) [upload=278214]: Got next segment. Inline: true Jul 7 20:33:36.423157427 (347h22m3.349452113s) [upload=278214]: Waiting for error group managing segments... Jul 7 20:33:36.423210227 (347h22m3.349504924s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 20:33:36.423217136 (347h22m3.349511831s) [upload=278214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 20:33:36.423221377 (347h22m3.349516075s) [upload=278214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 20:33:36.496833742 (347h22m3.423128455s) [upload=278214]: Done uploading segments... Jul 7 20:33:36.496854271 (347h22m3.423148945s) [upload=278214]: Upload finished. err: <nil> Duration: 248.289824ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)