Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 217.985204ms FAIL (2025-06-30 13:18:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 218.008815ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 238.642973ms FAIL (2025-06-30 13:17:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 238.686862ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 246.696411ms FAIL (2025-06-30 13:17:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 246.73595ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 239.443362ms FAIL (2025-06-30 13:16:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 239.550887ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 232.252545ms FAIL (2025-06-30 13:15:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 232.280401ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 240.585057ms FAIL (2025-06-30 13:14:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 240.617543ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 256.50037ms FAIL (2025-06-30 13:13:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 256.530657ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 228.700871ms FAIL (2025-06-30 13:13:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 228.734617ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 246.68577ms FAIL (2025-06-30 13:12:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 246.714718ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 272.21437ms FAIL (2025-06-30 13:11:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 272.233195ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 262.338198ms FAIL (2025-06-30 13:11:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 262.367881ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 358.992127ms FAIL (2025-06-30 13:10:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 359.015407ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 294.621439ms FAIL (2025-06-30 13:09:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 294.655531ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 292.359949ms FAIL (2025-06-30 13:09:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 292.399067ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 473.15187ms FAIL (2025-06-30 13:07:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 473.181518ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 417.875463ms FAIL (2025-06-30 13:06:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 417.913599ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 259.024988ms FAIL (2025-06-30 13:05:39 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 259.083712ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 274.000748ms FAIL (2025-06-30 13:05:03 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 274.054343ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 235.92608ms FAIL (2025-06-30 13:04:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 235.953466ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 391.063977ms FAIL (2025-06-30 13:03:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== d Jun 30 12:54:19.850138320 (171h42m46.776433013s) [upload=137567]: Done starting upload Jun 30 12:54:19.850241473 (171h42m46.776536145s) [upload=137567]: Uploading segments... Jun 30 12:54:19.850313152 (171h42m46.776607837s) [upload=137567]: Got next segment. Inline: true Jun 30 12:54:19.850318448 (171h42m46.77661312s) [upload=137567]: Waiting for error group managing segments... Jun 30 12:54:19.850368162 (171h42m46.776662835s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:54:19.850372886 (171h42m46.776667575s) [upload=137567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:54:19.850375400 (171h42m46.776670087s) [upload=137567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:54:19.911523047 (171h42m46.837817726s) [upload=137567]: Done uploading segments... Jun 30 12:54:19.911541941 (171h42m46.837836618s) [upload=137567]: Upload finished. err: <nil> Jun 30 12:55:14.438201539 (171h43m41.364496235s) [upload=137578]: Starting upload Jun 30 12:55:14.438325583 (171h43m41.364620279s) [upload=137578]: Done starting upload Jun 30 12:55:14.438440296 (171h43m41.364734972s) [upload=137578]: Uploading segments... Jun 30 12:55:14.438524918 (171h43m41.364819617s) [upload=137578]: Got next segment. Inline: true Jun 30 12:55:14.438531595 (171h43m41.364826287s) [upload=137578]: Waiting for error group managing segments... Jun 30 12:55:14.438607364 (171h43m41.364902051s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:55:14.438614483 (171h43m41.364909182s) [upload=137578]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:55:14.438637734 (171h43m41.364932427s) [upload=137578]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:55:14.507628041 (171h43m41.433922728s) [upload=137578]: Done uploading segments... Jun 30 12:55:14.507647379 (171h43m41.433942067s) [upload=137578]: Upload finished. err: <nil> Jun 30 12:56:28.469068898 (171h44m55.395363575s) [upload=137597]: Starting upload Jun 30 12:56:28.469154033 (171h44m55.39544872s) [upload=137597]: Done starting upload Jun 30 12:56:28.469274901 (171h44m55.395569581s) [upload=137597]: Uploading segments... Jun 30 12:56:28.469340887 (171h44m55.395635579s) [upload=137597]: Got next segment. Inline: true Jun 30 12:56:28.469347443 (171h44m55.395642145s) [upload=137597]: Waiting for error group managing segments... Jun 30 12:56:28.469395314 (171h44m55.395689989s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:28.469401443 (171h44m55.39569614s) [upload=137597]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:28.469420816 (171h44m55.395715511s) [upload=137597]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:28.535555260 (171h44m55.461849953s) [upload=137597]: Done uploading segments... Jun 30 12:56:28.535578413 (171h44m55.4618731s) [upload=137597]: Upload finished. err: <nil> Jun 30 12:56:59.517942990 (171h45m26.444237667s) [upload=137606]: Starting upload Jun 30 12:56:59.518029091 (171h45m26.444323777s) [upload=137606]: Done starting upload Jun 30 12:56:59.518101407 (171h45m26.444396087s) [upload=137606]: Uploading segments... Jun 30 12:56:59.518152328 (171h45m26.444447015s) [upload=137606]: Got next segment. Inline: true Jun 30 12:56:59.518157720 (171h45m26.444452396s) [upload=137606]: Waiting for error group managing segments... Jun 30 12:56:59.518203247 (171h45m26.44449793s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:56:59.518209643 (171h45m26.444504339s) [upload=137606]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:56:59.518213905 (171h45m26.444508599s) [upload=137606]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:56:59.578374956 (171h45m26.504669634s) [upload=137606]: Done uploading segments... Jun 30 12:56:59.578393825 (171h45m26.504688501s) [upload=137606]: Upload finished. err: <nil> Jun 30 12:58:04.707109173 (171h46m31.633403874s) [upload=137618]: Starting upload Jun 30 12:58:04.707213400 (171h46m31.633508077s) [upload=137618]: Done starting upload Jun 30 12:58:04.707318070 (171h46m31.633612756s) [upload=137618]: Uploading segments... Jun 30 12:58:04.707406228 (171h46m31.633700924s) [upload=137618]: Got next segment. Inline: true Jun 30 12:58:04.707414472 (171h46m31.633709162s) [upload=137618]: Waiting for error group managing segments... Jun 30 12:58:04.707474429 (171h46m31.633769131s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:04.707483954 (171h46m31.633778647s) [upload=137618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:04.707487985 (171h46m31.633782659s) [upload=137618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:04.767340193 (171h46m31.693634871s) [upload=137618]: Done uploading segments... Jun 30 12:58:04.767359969 (171h46m31.693654645s) [upload=137618]: Upload finished. err: <nil> Jun 30 12:58:50.949783376 (171h47m17.876078066s) [upload=137628]: Starting upload Jun 30 12:58:50.949880041 (171h47m17.876174732s) [upload=137628]: Done starting upload Jun 30 12:58:50.949933739 (171h47m17.876228446s) [upload=137628]: Uploading segments... Jun 30 12:58:50.950028886 (171h47m17.876323601s) [upload=137628]: Got next segment. Inline: true Jun 30 12:58:50.950035168 (171h47m17.876329856s) [upload=137628]: Waiting for error group managing segments... Jun 30 12:58:50.950106062 (171h47m17.876400749s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:58:50.950112055 (171h47m17.876406745s) [upload=137628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:58:50.950115378 (171h47m17.876410067s) [upload=137628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:58:51.011323384 (171h47m17.937618071s) [upload=137628]: Done uploading segments... Jun 30 12:58:51.011341547 (171h47m17.937636233s) [upload=137628]: Upload finished. err: <nil> Jun 30 12:59:34.511701359 (171h48m1.437996035s) [upload=137636]: Starting upload Jun 30 12:59:34.511772591 (171h48m1.438067281s) [upload=137636]: Done starting upload Jun 30 12:59:34.511851823 (171h48m1.438146503s) [upload=137636]: Uploading segments... Jun 30 12:59:34.511902891 (171h48m1.438197587s) [upload=137636]: Got next segment. Inline: true Jun 30 12:59:34.511909364 (171h48m1.438204053s) [upload=137636]: Waiting for error group managing segments... Jun 30 12:59:34.511979666 (171h48m1.438274356s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 12:59:34.511986952 (171h48m1.438281642s) [upload=137636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 12:59:34.511991804 (171h48m1.438286495s) [upload=137636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 12:59:34.592474584 (171h48m1.518769271s) [upload=137636]: Done uploading segments... Jun 30 12:59:34.592493677 (171h48m1.518788365s) [upload=137636]: Upload finished. err: <nil> Jun 30 13:00:11.879387252 (171h48m38.805681951s) [upload=137646]: Starting upload Jun 30 13:00:11.879540320 (171h48m38.80583502s) [upload=137646]: Done starting upload Jun 30 13:00:11.879694893 (171h48m38.805989587s) [upload=137646]: Uploading segments... Jun 30 13:00:11.879754809 (171h48m38.806049537s) [upload=137646]: Got next segment. Inline: true Jun 30 13:00:11.879763286 (171h48m38.806057987s) [upload=137646]: Waiting for error group managing segments... Jun 30 13:00:11.879824566 (171h48m38.806119266s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:11.879832262 (171h48m38.806126962s) [upload=137646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:11.879837709 (171h48m38.806132408s) [upload=137646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:11.958618186 (171h48m38.884912881s) [upload=137646]: Done uploading segments... Jun 30 13:00:11.958656359 (171h48m38.88495105s) [upload=137646]: Upload finished. err: <nil> Jun 30 13:00:50.309066462 (171h49m17.235361161s) [upload=137657]: Starting upload Jun 30 13:00:50.309174902 (171h49m17.235469599s) [upload=137657]: Done starting upload Jun 30 13:00:50.309302655 (171h49m17.235597328s) [upload=137657]: Uploading segments... Jun 30 13:00:50.309368898 (171h49m17.235663596s) [upload=137657]: Got next segment. Inline: true Jun 30 13:00:50.309376118 (171h49m17.235670806s) [upload=137657]: Waiting for error group managing segments... Jun 30 13:00:50.309435031 (171h49m17.23572973s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:00:50.309443084 (171h49m17.235737783s) [upload=137657]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:00:50.309448071 (171h49m17.235742764s) [upload=137657]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:00:50.367871760 (171h49m17.294166502s) [upload=137657]: Done uploading segments... Jun 30 13:00:50.367898080 (171h49m17.294192774s) [upload=137657]: Upload finished. err: <nil> Jun 30 13:01:24.497976188 (171h49m51.424270881s) [upload=137664]: Starting upload Jun 30 13:01:24.498096801 (171h49m51.424391499s) [upload=137664]: Done starting upload Jun 30 13:01:24.498224405 (171h49m51.424519094s) [upload=137664]: Uploading segments... Jun 30 13:01:24.498280720 (171h49m51.424575419s) [upload=137664]: Got next segment. Inline: true Jun 30 13:01:24.498287957 (171h49m51.424582647s) [upload=137664]: Waiting for error group managing segments... Jun 30 13:01:24.498343604 (171h49m51.424638282s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 30 13:01:24.498350223 (171h49m51.424644899s) [upload=137664]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 30 13:01:24.498354841 (171h49m51.424649519s) [upload=137664]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 30 13:01:24.590965607 (171h49m51.517260306s) [upload=137664]: Done uploading segments... Jun 30 13:01:24.590989415 (171h49m51.51728411s) [upload=137664]: Upload finished. err: <nil> Duration: 391.083539ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 270.80514ms FAIL (2025-06-30 11:30:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 270.83471ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 341.008554ms FAIL (2025-06-30 11:30:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 341.029965ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 331.535155ms FAIL (2025-06-30 11:29:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 331.566622ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 291.46178ms FAIL (2025-06-30 11:28:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 291.482312ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 281.960502ms FAIL (2025-06-30 11:28:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 281.986587ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 282.318067ms FAIL (2025-06-30 11:27:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 282.353981ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 276.932105ms FAIL (2025-06-30 11:27:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 277.021792ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 525.040787ms FAIL (2025-06-30 11:26:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 526.23166ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 252.299503ms FAIL (2025-06-30 11:25:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 252.332463ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 255.78892ms FAIL (2025-06-30 11:24:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 255.81991ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 267.406638ms FAIL (2025-06-30 11:24:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 267.457518ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 258.090029ms FAIL (2025-06-30 11:23:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 258.15711ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 296.442096ms FAIL (2025-06-30 11:22:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 296.467071ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 275.137329ms FAIL (2025-06-30 11:21:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 275.158667ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 248.027047ms FAIL (2025-06-30 11:21:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 248.095364ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 315.64159ms FAIL (2025-06-30 11:20:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 315.663647ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 268.629255ms FAIL (2025-06-30 11:20:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 268.66411ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 476.193867ms FAIL (2025-06-30 11:19:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.219822ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 275.857768ms FAIL (2025-06-30 11:18:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 275.880508ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 285.800296ms FAIL (2025-06-30 11:17:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 285.863434ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 267.7929ms FAIL (2025-06-30 11:17:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 267.817352ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 470.87123ms FAIL (2025-06-30 11:15:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 470.919446ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 270.14119ms FAIL (2025-06-30 11:15:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 270.160118ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 812.718535ms FAIL (2025-06-30 11:14:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 812.741411ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 303.155232ms FAIL (2025-06-30 11:14:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 303.175469ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 303.609843ms FAIL (2025-06-30 11:13:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 303.659862ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 277.47209ms FAIL (2025-06-30 11:12:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 277.506745ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 341.852775ms FAIL (2025-06-30 11:11:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 341.883372ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 386.148019ms FAIL (2025-06-30 11:10:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 386.172641ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 631.290583ms FAIL (2025-06-30 11:09:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 631.313192ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)