Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
249.491153ms
FAIL (2025-07-04 17:24:49 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 249.526193ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
243.461129ms
FAIL (2025-07-04 17:24:04 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 243.489453ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
255.234386ms
FAIL (2025-07-04 17:23:33 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 255.301825ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
759.383115ms
FAIL (2025-07-04 17:23:10 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 759.503952ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
245.197783ms
FAIL (2025-07-04 17:22:54 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 245.224408ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
236.565104ms
FAIL (2025-07-04 17:22:12 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 236.675317ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
212.750991ms
FAIL (2025-07-04 17:21:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 212.776158ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
293.081558ms
FAIL (2025-07-04 17:21:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 293.127329ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
270.378572ms
FAIL (2025-07-04 17:20:10 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 270.434279ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
256.053171ms
FAIL (2025-07-04 17:19:10 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 256.103559ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
245.368239ms
FAIL (2025-07-04 17:18:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 245.399007ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
383.982486ms
FAIL (2025-07-04 17:17:16 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 384.010766ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
260.029962ms
FAIL (2025-07-04 17:16:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 260.063941ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
229.307546ms
FAIL (2025-07-04 17:16:16 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 229.335115ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
261.38934ms
FAIL (2025-07-04 17:15:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 261.422907ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
415.731493ms
FAIL (2025-07-04 17:14:16 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 415.757187ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
260.05308ms
FAIL (2025-07-04 17:13:05 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 260.084602ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
255.156881ms
FAIL (2025-07-04 17:11:56 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 255.182904ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
228.791149ms
FAIL (2025-07-04 17:11:28 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 228.823642ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
273.378395ms
FAIL (2025-07-04 17:11:07 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 273.408003ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
262.273417ms
FAIL (2025-07-04 17:10:22 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 262.320637ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
243.896643ms
FAIL (2025-07-04 17:09:33 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 243.92543ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
247.566215ms
FAIL (2025-07-04 17:08:39 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 247.593993ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
245.682624ms
FAIL (2025-07-04 17:08:05 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 245.708388ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
275.704565ms
FAIL (2025-07-04 17:07:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 275.732962ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
276.79496ms
FAIL (2025-07-04 17:06:34 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 276.820048ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
267.91142ms
FAIL (2025-07-04 17:05:34 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 267.971471ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
236.951104ms
FAIL (2025-07-04 17:04:58 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 236.972755ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
246.372631ms
FAIL (2025-07-04 17:04:29 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 246.411652ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
268.339206ms
FAIL (2025-07-04 17:03:55 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 268.394872ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
382.133545ms
FAIL (2025-07-04 17:03:07 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 382.164167ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001225824s
FAIL (2025-07-04 17:03:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== l 4 16:54:24.446974089 (271h42m51.373268785s) [upload=217578]: Done starting upload Jul 4 16:54:24.447075250 (271h42m51.373369922s) [upload=217578]: Uploading segments... Jul 4 16:54:24.447176120 (271h42m51.373470814s) [upload=217578]: Got next segment. Inline: true Jul 4 16:54:24.447181635 (271h42m51.373476312s) [upload=217578]: Waiting for error group managing segments... Jul 4 16:54:24.447223385 (271h42m51.373518062s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:24.447228416 (271h42m51.373523102s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:24.447231590 (271h42m51.373526277s) [upload=217578]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:24.547434376 (271h42m51.473729073s) [upload=217578]: Done uploading segments... Jul 4 16:54:24.547461753 (271h42m51.473756443s) [upload=217578]: Upload finished. err: <nil> Jul 4 16:54:50.012334956 (271h43m16.938629643s) [upload=217583]: Starting upload Jul 4 16:54:50.012433543 (271h43m16.938728215s) [upload=217583]: Done starting upload Jul 4 16:54:50.012522785 (271h43m16.938817456s) [upload=217583]: Uploading segments... Jul 4 16:54:50.012591678 (271h43m16.938886376s) [upload=217583]: Got next segment. Inline: true Jul 4 16:54:50.012598154 (271h43m16.938892842s) [upload=217583]: Waiting for error group managing segments... Jul 4 16:54:50.012662309 (271h43m16.938957005s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:54:50.012668193 (271h43m16.938962881s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:54:50.012671924 (271h43m16.938966643s) [upload=217583]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:54:50.084069967 (271h43m17.010364644s) [upload=217583]: Done uploading segments... Jul 4 16:54:50.084093930 (271h43m17.010388611s) [upload=217583]: Upload finished. err: <nil> Jul 4 16:55:34.728175858 (271h44m1.654470546s) [upload=217594]: Starting upload Jul 4 16:55:34.728267786 (271h44m1.654562463s) [upload=217594]: Done starting upload Jul 4 16:55:34.728356170 (271h44m1.654650849s) [upload=217594]: Uploading segments... Jul 4 16:55:34.728396261 (271h44m1.654690957s) [upload=217594]: Got next segment. Inline: true Jul 4 16:55:34.728401326 (271h44m1.654696014s) [upload=217594]: Waiting for error group managing segments... Jul 4 16:55:34.728458939 (271h44m1.654753627s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:55:34.728465387 (271h44m1.654760082s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:55:34.728469617 (271h44m1.654764311s) [upload=217594]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:55:34.787990513 (271h44m1.714285196s) [upload=217594]: Done uploading segments... Jul 4 16:55:34.788012542 (271h44m1.714307218s) [upload=217594]: Upload finished. err: <nil> Jul 4 16:56:22.832810212 (271h44m49.759104889s) [upload=217605]: Starting upload Jul 4 16:56:22.832893787 (271h44m49.759188475s) [upload=217605]: Done starting upload Jul 4 16:56:22.832970874 (271h44m49.759265552s) [upload=217605]: Uploading segments... Jul 4 16:56:22.833039764 (271h44m49.759334441s) [upload=217605]: Got next segment. Inline: true Jul 4 16:56:22.833044960 (271h44m49.759339636s) [upload=217605]: Waiting for error group managing segments... Jul 4 16:56:22.833137558 (271h44m49.759432234s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:56:22.833153249 (271h44m49.759447925s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:56:22.833157111 (271h44m49.759451787s) [upload=217605]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:56:23.004401233 (271h44m49.930695931s) [upload=217605]: Done uploading segments... Jul 4 16:56:23.004419935 (271h44m49.930714625s) [upload=217605]: Upload finished. err: <nil> Jul 4 16:57:14.763915698 (271h45m41.690210394s) [upload=217615]: Starting upload Jul 4 16:57:14.764018476 (271h45m41.690313164s) [upload=217615]: Done starting upload Jul 4 16:57:14.764146620 (271h45m41.690441308s) [upload=217615]: Uploading segments... Jul 4 16:57:14.764203672 (271h45m41.690498372s) [upload=217615]: Got next segment. Inline: true Jul 4 16:57:14.764209947 (271h45m41.69050464s) [upload=217615]: Waiting for error group managing segments... Jul 4 16:57:14.764264588 (271h45m41.690559282s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:57:14.764272240 (271h45m41.690566938s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:57:14.764276629 (271h45m41.690571324s) [upload=217615]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:57:14.839328938 (271h45m41.765623632s) [upload=217615]: Done uploading segments... Jul 4 16:57:14.839353527 (271h45m41.765648218s) [upload=217615]: Upload finished. err: <nil> Jul 4 16:58:13.609116264 (271h46m40.535410958s) [upload=217628]: Starting upload Jul 4 16:58:13.609216934 (271h46m40.535511635s) [upload=217628]: Done starting upload Jul 4 16:58:13.609323370 (271h46m40.535618043s) [upload=217628]: Uploading segments... Jul 4 16:58:13.609482428 (271h46m40.535777123s) [upload=217628]: Got next segment. Inline: true Jul 4 16:58:13.609490024 (271h46m40.535784724s) [upload=217628]: Waiting for error group managing segments... Jul 4 16:58:13.609565286 (271h46m40.535859984s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:58:13.609588632 (271h46m40.535883321s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:58:13.609593329 (271h46m40.535888026s) [upload=217628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:58:13.714383856 (271h46m40.640678532s) [upload=217628]: Done uploading segments... Jul 4 16:58:13.714402167 (271h46m40.640696843s) [upload=217628]: Upload finished. err: <nil> Jul 4 16:59:15.655568095 (271h47m42.581862791s) [upload=217643]: Starting upload Jul 4 16:59:15.655648174 (271h47m42.58194286s) [upload=217643]: Done starting upload Jul 4 16:59:15.655768989 (271h47m42.582063668s) [upload=217643]: Uploading segments... Jul 4 16:59:15.655816161 (271h47m42.582110857s) [upload=217643]: Got next segment. Inline: true Jul 4 16:59:15.655821399 (271h47m42.582116086s) [upload=217643]: Waiting for error group managing segments... Jul 4 16:59:15.655882464 (271h47m42.582177151s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 16:59:15.655887954 (271h47m42.58218264s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 16:59:15.655891593 (271h47m42.582186281s) [upload=217643]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 16:59:15.714639935 (271h47m42.64093462s) [upload=217643]: Done uploading segments... Jul 4 16:59:15.714662737 (271h47m42.640957424s) [upload=217643]: Upload finished. err: <nil> Jul 4 17:00:14.409573054 (271h48m41.335867746s) [upload=217652]: Starting upload Jul 4 17:00:14.409719167 (271h48m41.336013857s) [upload=217652]: Done starting upload Jul 4 17:00:14.409888828 (271h48m41.336183517s) [upload=217652]: Uploading segments... Jul 4 17:00:14.409983947 (271h48m41.336278646s) [upload=217652]: Got next segment. Inline: true Jul 4 17:00:14.409991581 (271h48m41.336286278s) [upload=217652]: Waiting for error group managing segments... Jul 4 17:00:14.410085931 (271h48m41.336380621s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:14.410092746 (271h48m41.336387445s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:14.410096964 (271h48m41.336391663s) [upload=217652]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:14.532446409 (271h48m41.45874109s) [upload=217652]: Done uploading segments... Jul 4 17:00:14.532470733 (271h48m41.458765407s) [upload=217652]: Upload finished. err: <nil> Jul 4 17:00:45.857211442 (271h49m12.783506139s) [upload=217661]: Starting upload Jul 4 17:00:45.857307182 (271h49m12.783601872s) [upload=217661]: Done starting upload Jul 4 17:00:45.857382792 (271h49m12.783677464s) [upload=217661]: Uploading segments... Jul 4 17:00:45.857434180 (271h49m12.783728878s) [upload=217661]: Got next segment. Inline: true Jul 4 17:00:45.857442206 (271h49m12.783736906s) [upload=217661]: Waiting for error group managing segments... Jul 4 17:00:45.857496115 (271h49m12.783790805s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:00:45.857502240 (271h49m12.783796936s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:00:45.857506169 (271h49m12.783800861s) [upload=217661]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:00:45.931783557 (271h49m12.85807828s) [upload=217661]: Done uploading segments... Jul 4 17:00:45.931806872 (271h49m12.858101559s) [upload=217661]: Upload finished. err: <nil> Jul 4 17:01:53.515875549 (271h50m20.442170236s) [upload=217673]: Starting upload Jul 4 17:01:53.515960925 (271h50m20.442255601s) [upload=217673]: Done starting upload Jul 4 17:01:53.516035601 (271h50m20.442330274s) [upload=217673]: Uploading segments... Jul 4 17:01:53.516081946 (271h50m20.442376645s) [upload=217673]: Got next segment. Inline: true Jul 4 17:01:53.516088284 (271h50m20.442382976s) [upload=217673]: Waiting for error group managing segments... Jul 4 17:01:53.516142954 (271h50m20.442437651s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 17:01:53.516149778 (271h50m20.442444473s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 17:01:53.516153718 (271h50m20.442448415s) [upload=217673]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 17:01:53.599167024 (271h50m20.525461721s) [upload=217673]: Done uploading segments... Jul 4 17:01:53.599188431 (271h50m20.525483127s) [upload=217673]: Upload finished. err: <nil>
Duration: 30.001264893s
core
uplink
us-select-1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
223.818063ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 16:03:57 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 223.860643ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
216.40848ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 16:02:43 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 216.432481ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
228.967567ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 16:01:54 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 228.97924ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
241.271153ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 16:01:30 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 241.29553ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
271.166921ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 16:00:58 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 271.199914ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
245.670003ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 16:00:25 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 245.762281ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
346.870455ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:59:26 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 346.897345ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
234.603926ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:58:14 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 234.630683ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.354750837s remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:57:35 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 1.354794553s
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
226.827232ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:56:59 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 226.847631ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
220.307483ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:56:00 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 220.387698ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
307.151728ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:54:58 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 307.174876ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
259.632078ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:54:40 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 259.655411ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
385.497911ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:54:16 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 385.528056ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
247.765628ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:53:05 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 247.787531ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
253.453932ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:52:31 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 253.484075ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
221.24705ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:51:52 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 221.271086ms
edge
ipfs
us1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
219.368201ms remote:
"tcp:5.161.37.179:443" response headers:
Access-Control-Allow-Origin=* Content-Length=0 Date=Fri, 04 Jul 2025 15:19:23 GMT target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-04 15:51:15 UTC)
orbiter warning: unexpected status code "502 Bad Gateway" ""
Duration: 219.391313ms
edge
ipfs
us1
|