Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [....................X.XXX.XX................................] duration: 610.139874ms FAIL (2025-08-07 23:40:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 610.244164ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 51.692887ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:40:53 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 51.718236ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................XXXXX.................................] duration: 1.816845889s FAIL (2025-08-07 23:40:51 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.816873645s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....................XX..X...X..............................] duration: 260.682342ms FAIL (2025-08-07 23:40:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 162725354 (13h31m20.67782793s) [upload=13919]: Starting upload Aug 7 23:10:46.162807728 (13h31m20.677910304s) [upload=13919]: Done starting upload Aug 7 23:10:46.162877771 (13h31m20.677980347s) [upload=13919]: Uploading segments... Aug 7 23:10:46.162939319 (13h31m20.678041901s) [upload=13919]: Got next segment. Inline: true Aug 7 23:10:46.162944044 (13h31m20.678046624s) [upload=13919]: Waiting for error group managing segments... Aug 7 23:10:46.162986704 (13h31m20.678089286s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:46.162991049 (13h31m20.678093629s) [upload=13919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:46.162994213 (13h31m20.678096792s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:49.331481110 (13h31m23.84658369s) [upload=13919]: Done uploading segments... Aug 7 23:10:49.331502710 (13h31m23.846605285s) [upload=13919]: Upload finished. err: <nil> Aug 7 23:11:56.588799333 (13h32m31.103901909s) [upload=13942]: Starting upload Aug 7 23:11:56.588871172 (13h32m31.103973747s) [upload=13942]: Done starting upload Aug 7 23:11:56.588959193 (13h32m31.104061755s) [upload=13942]: Uploading segments... Aug 7 23:11:56.588996486 (13h32m31.104099072s) [upload=13942]: Got next segment. Inline: true Aug 7 23:11:56.589000914 (13h32m31.104103491s) [upload=13942]: Waiting for error group managing segments... Aug 7 23:11:56.589047870 (13h32m31.104150433s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:56.589051856 (13h32m31.104154432s) [upload=13942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:56.589054372 (13h32m31.10415695s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:56.555383755 (13h33m31.070486344s) [upload=13942]: Done uploading segments... Aug 7 23:12:56.555407795 (13h33m31.070510378s) [upload=13942]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:56.669023639 (13h33m31.184126201s) [upload=13959]: Starting upload Aug 7 23:12:56.669081512 (13h33m31.184184088s) [upload=13959]: Done starting upload Aug 7 23:12:56.669139265 (13h33m31.184241827s) [upload=13959]: Uploading segments... Aug 7 23:12:56.669181725 (13h33m31.184284307s) [upload=13959]: Got next segment. Inline: true Aug 7 23:12:56.669185727 (13h33m31.18428829s) [upload=13959]: Waiting for error group managing segments... Aug 7 23:12:56.669222629 (13h33m31.184325191s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:56.669226085 (13h33m31.18432866s) [upload=13959]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:56.669228556 (13h33m31.184331118s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.639927224 (13h34m31.155029791s) [upload=13959]: Done uploading segments... Aug 7 23:13:56.639946863 (13h34m31.155049429s) [upload=13959]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:56.754611797 (13h34m31.269714367s) [upload=13974]: Starting upload Aug 7 23:13:56.754690544 (13h34m31.269793123s) [upload=13974]: Done starting upload Aug 7 23:13:56.754787910 (13h34m31.269890478s) [upload=13974]: Uploading segments... Aug 7 23:13:56.754829279 (13h34m31.269931864s) [upload=13974]: Got next segment. Inline: true Aug 7 23:13:56.754833834 (13h34m31.269936411s) [upload=13974]: Waiting for error group managing segments... Aug 7 23:13:56.754887650 (13h34m31.269990217s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.754891993 (13h34m31.26999456s) [upload=13974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.754895495 (13h34m31.269998071s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.814228870 (13h34m31.32933145s) [upload=13974]: Done uploading segments... Aug 7 23:13:56.814253732 (13h34m31.329356295s) [upload=13974]: Upload finished. err: <nil> Aug 7 23:14:51.237083244 (13h35m25.75218581s) [upload=13984]: Starting upload Aug 7 23:14:51.237154440 (13h35m25.752257016s) [upload=13984]: Done starting upload Aug 7 23:14:51.237222602 (13h35m25.752325168s) [upload=13984]: Uploading segments... Aug 7 23:14:51.237272573 (13h35m25.752375158s) [upload=13984]: Got next segment. Inline: true Aug 7 23:14:51.237277088 (13h35m25.752379654s) [upload=13984]: Waiting for error group managing segments... Aug 7 23:14:51.237328372 (13h35m25.752430937s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:51.237332974 (13h35m25.75243555s) [upload=13984]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:51.237336117 (13h35m25.752438684s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:51.367347705 (13h35m25.882450281s) [upload=13984]: Done uploading segments... Aug 7 23:14:51.367368302 (13h35m25.882470865s) [upload=13984]: Upload finished. err: <nil> Aug 7 23:16:05.237199068 (13h36m39.752301655s) [upload=14002]: Starting upload Aug 7 23:16:05.237269888 (13h36m39.752372457s) [upload=14002]: Done starting upload Aug 7 23:16:05.237360382 (13h36m39.752462952s) [upload=14002]: Uploading segments... Aug 7 23:16:05.237841750 (13h36m39.752944336s) [upload=14002]: Got next segment. Inline: true Aug 7 23:16:05.237859632 (13h36m39.752962218s) [upload=14002]: Waiting for error group managing segments... Aug 7 23:16:05.237930004 (13h36m39.753032586s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:05.237936533 (13h36m39.753039109s) [upload=14002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:05.237940573 (13h36m39.753043148s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:05.205359206 (13h37m39.720461782s) [upload=14002]: Done uploading segments... Aug 7 23:17:05.205376470 (13h37m39.720479046s) [upload=14002]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:15.921193047 (13h37m50.436295623s) [upload=14020]: Starting upload Aug 7 23:17:15.921258731 (13h37m50.436361294s) [upload=14020]: Done starting upload Aug 7 23:17:15.921299906 (13h37m50.436402469s) [upload=14020]: Uploading segments... Aug 7 23:17:15.921331213 (13h37m50.436433799s) [upload=14020]: Got next segment. Inline: true Aug 7 23:17:15.921334685 (13h37m50.436437248s) [upload=14020]: Waiting for error group managing segments... Aug 7 23:17:15.921366597 (13h37m50.436469172s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:15.921370316 (13h37m50.436472878s) [upload=14020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:15.921372605 (13h37m50.436475167s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:15.996901858 (13h37m50.512004437s) [upload=14020]: Done uploading segments... Aug 7 23:17:15.996920869 (13h37m50.512023431s) [upload=14020]: Upload finished. err: <nil> Aug 7 23:18:10.846312036 (13h38m45.361414612s) [upload=14034]: Starting upload Aug 7 23:18:10.846377244 (13h38m45.36147981s) [upload=14034]: Done starting upload Aug 7 23:18:10.846461339 (13h38m45.361563904s) [upload=14034]: Uploading segments... Aug 7 23:18:10.846496473 (13h38m45.361599058s) [upload=14034]: Got next segment. Inline: true Aug 7 23:18:10.846501164 (13h38m45.36160374s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:18:10.846554786 (13h38m45.361657362s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:10.846559294 (13h38m45.361661871s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:10.846562922 (13h38m45.361665498s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:11.200018957 (13h38m45.715121534s) [upload=14034]: Done uploading segments... Aug 7 23:18:11.200049582 (13h38m45.715152158s) [upload=14034]: Upload finished. err: <nil> Aug 7 23:19:15.896167300 (13h39m50.411269878s) [upload=14051]: Starting upload Aug 7 23:19:15.896353995 (13h39m50.411456637s) [upload=14051]: Done starting upload Aug 7 23:19:15.896436268 (13h39m50.411538832s) [upload=14051]: Uploading segments... Aug 7 23:19:15.896501153 (13h39m50.411603732s) [upload=14051]: Got next segment. Inline: true Aug 7 23:19:15.896505978 (13h39m50.411608555s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:19:15.896570054 (13h39m50.411672631s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:15.896574888 (13h39m50.411677465s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:15.896578764 (13h39m50.411681341s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:15.976040109 (13h39m50.491142676s) [upload=14051]: Done uploading segments... Aug 7 23:19:15.976057062 (13h39m50.491159637s) [upload=14051]: Upload finished. err: <nil> Aug 7 23:19:50.253965481 (13h40m24.76906806s) [upload=14062]: Starting upload Aug 7 23:19:50.254050632 (13h40m24.769153219s) [upload=14062]: Done starting upload Aug 7 23:19:50.254136562 (13h40m24.769239133s) [upload=14062]: Uploading segments... Aug 7 23:19:50.254189310 (13h40m24.769291896s) [upload=14062]: Got next segment. Inline: true Aug 7 23:19:50.254194507 (13h40m24.769297084s) [upload=14062]: Waiting for error group managing segments... Aug 7 23:19:50.254255974 (13h40m24.769358555s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:50.254262645 (13h40m24.769365222s) [upload=14062]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:50.254267031 (13h40m24.769369607s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.224097554 (13h41m24.739200131s) [upload=14062]: Done uploading segments... Aug 7 23:20:50.224116508 (13h41m24.739219086s) [upload=14062]: Upload finished. err: metaclient: context deadline exceeded Duration: 260.701007ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......................XX.XXX...............................] duration: 243.226589ms FAIL (2025-08-07 23:40:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 30m31.135428317s) [upload=13902]: Starting upload Aug 7 23:09:56.620515140 (13h30m31.135617722s) [upload=13902]: Done starting upload Aug 7 23:09:56.620600541 (13h30m31.135703106s) [upload=13902]: Uploading segments... Aug 7 23:09:56.620663979 (13h30m31.135766559s) [upload=13902]: Got next segment. Inline: true Aug 7 23:09:56.620669313 (13h30m31.13577189s) [upload=13902]: Waiting for error group managing segments... Aug 7 23:09:56.620725144 (13h30m31.13582772s) [upload=13902]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:09:56.620729700 (13h30m31.135832276s) [upload=13902]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:09:56.620732858 (13h30m31.135835435s) [upload=13902]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:56.756931332 (13h30m31.272033912s) [upload=13902]: Done uploading segments... Aug 7 23:09:56.756952203 (13h30m31.272054766s) [upload=13902]: Upload finished. err: <nil> Aug 7 23:10:22.255312070 (13h30m56.770414636s) [upload=13913]: Starting upload Aug 7 23:10:22.255389033 (13h30m56.770491609s) [upload=13913]: Done starting upload Aug 7 23:10:22.255538785 (13h30m56.77064136s) [upload=13913]: Uploading segments... Aug 7 23:10:22.255632108 (13h30m56.77073469s) [upload=13913]: Got next segment. Inline: true Aug 7 23:10:22.255639107 (13h30m56.770741683s) [upload=13913]: Waiting for error group managing segments... Aug 7 23:10:22.255686584 (13h30m56.77078915s) [upload=13913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:22.255691754 (13h30m56.770794331s) [upload=13913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:22.255695470 (13h30m56.770798046s) [upload=13913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:22.322045125 (13h30m56.837147702s) [upload=13913]: Done uploading segments... Aug 7 23:10:22.322068753 (13h30m56.837171329s) [upload=13913]: Upload finished. err: <nil> Aug 7 23:11:00.342239204 (13h31m34.85734177s) [upload=13922]: Starting upload Aug 7 23:11:00.342310667 (13h31m34.857413235s) [upload=13922]: Done starting upload Aug 7 23:11:00.342383830 (13h31m34.85748641s) [upload=13922]: Uploading segments... Aug 7 23:11:00.342449267 (13h31m34.857551854s) [upload=13922]: Got next segment. Inline: true Aug 7 23:11:00.342455504 (13h31m34.857558092s) [upload=13922]: Waiting for error group managing segments... Aug 7 23:11:00.342503627 (13h31m34.857606209s) [upload=13922]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:00.342508465 (13h31m34.857611046s) [upload=13922]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:00.342511859 (13h31m34.857614451s) [upload=13922]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:00.401913127 (13h31m34.917015712s) [upload=13922]: Done uploading segments... Aug 7 23:11:00.401931904 (13h31m34.917034472s) [upload=13922]: Upload finished. err: <nil> Aug 7 23:11:35.523175401 (13h32m10.03827798s) [upload=13934]: Starting upload Aug 7 23:11:35.523266566 (13h32m10.038369147s) [upload=13934]: Done starting upload Aug 7 23:11:35.523355068 (13h32m10.038457647s) [upload=13934]: Uploading segments... Aug 7 23:11:35.523405664 (13h32m10.03850825s) [upload=13934]: Got next segment. Inline: true Aug 7 23:11:35.523412393 (13h32m10.038514977s) [upload=13934]: Waiting for error group managing segments... Aug 7 23:11:35.523470368 (13h32m10.038572947s) [upload=13934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:35.523479499 (13h32m10.038582083s) [upload=13934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:35.523484240 (13h32m10.038586821s) [upload=13934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:35.587656651 (13h32m10.102759216s) [upload=13934]: Done uploading segments... Aug 7 23:11:35.587674771 (13h32m10.102777338s) [upload=13934]: Upload finished. err: <nil> Aug 7 23:12:23.226340957 (13h32m57.741443533s) [upload=13947]: Starting upload Aug 7 23:12:23.226411041 (13h32m57.741513618s) [upload=13947]: Done starting upload Aug 7 23:12:23.226470575 (13h32m57.74157314s) [upload=13947]: Uploading segments... Aug 7 23:12:23.226506745 (13h32m57.741609326s) [upload=13947]: Got next segment. Inline: true Aug 7 23:12:23.226511437 (13h32m57.741614019s) [upload=13947]: Waiting for error group managing segments... Aug 7 23:12:23.226561258 (13h32m57.741663839s) [upload=13947]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:23.226566182 (13h32m57.741668762s) [upload=13947]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:23.226569380 (13h32m57.741671956s) [upload=13947]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:53.198277272 (13h33m27.713379848s) [upload=13947]: Done uploading segments... Aug 7 23:12:53.198295252 (13h33m27.713397814s) [upload=13947]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:53.310207725 (13h33m27.825310302s) [upload=13957]: Starting upload Aug 7 23:12:53.310290257 (13h33m27.82539282s) [upload=13957]: Done starting upload Aug 7 23:12:53.310379699 (13h33m27.825482276s) [upload=13957]: Uploading segments... Aug 7 23:12:53.310488747 (13h33m27.825591336s) [upload=13957]: Got next segment. Inline: true Aug 7 23:12:53.310497967 (13h33m27.825600548s) [upload=13957]: Waiting for error group managing segments... Aug 7 23:12:53.310564975 (13h33m27.825667562s) [upload=13957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:53.310572430 (13h33m27.825675017s) [upload=13957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:53.310576051 (13h33m27.825678628s) [upload=13957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:23.280527765 (13h33m57.79563034s) [upload=13957]: Done uploading segments... Aug 7 23:13:23.280561185 (13h33m57.795663762s) [upload=13957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:50.810735096 (13h34m25.325837673s) [upload=13972]: Starting upload Aug 7 23:13:50.810789852 (13h34m25.325892437s) [upload=13972]: Done starting upload Aug 7 23:13:50.810850987 (13h34m25.325953556s) [upload=13972]: Uploading segments... Aug 7 23:13:50.810923216 (13h34m25.326025797s) [upload=13972]: Got next segment. Inline: true Aug 7 23:13:50.810928523 (13h34m25.326031096s) [upload=13972]: Waiting for error group managing segments... Aug 7 23:13:50.810970440 (13h34m25.326073016s) [upload=13972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:50.810974710 (13h34m25.326077286s) [upload=13972]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:50.810978274 (13h34m25.32608085s) [upload=13972]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:50.866577223 (13h34m25.3816798s) [upload=13972]: Done uploading segments... Aug 7 23:13:50.866594654 (13h34m25.38169723s) [upload=13972]: Upload finished. err: <nil> Aug 7 23:14:54.225692687 (13h35m28.740795254s) [upload=13986]: Starting upload Aug 7 23:14:54.225785134 (13h35m28.740887721s) [upload=13986]: Done starting upload Aug 7 23:14:54.225870963 (13h35m28.740973529s) [upload=13986]: Uploading segments... Aug 7 23:14:54.225923809 (13h35m28.741026386s) [upload=13986]: Got next segment. Inline: true Aug 7 23:14:54.225928631 (13h35m28.741031207s) [upload=13986]: Waiting for error group managing segments... Aug 7 23:14:54.225969273 (13h35m28.741071842s) [upload=13986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:54.225973734 (13h35m28.7410763s) [upload=13986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:54.225976991 (13h35m28.741079566s) [upload=13986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:24.191802298 (13h35m58.706904867s) [upload=13986]: Done uploading segments... Aug 7 23:15:24.191822370 (13h35m58.70692494s) [upload=13986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:24.446619359 (13h35m58.961721924s) [upload=13992]: Starting upload Aug 7 23:15:24.446725754 (13h35m58.961828338s) [upload=13992]: Done starting upload Aug 7 23:15:24.446812887 (13h35m58.961915463s) [upload=13992]: Uploading segments... Aug 7 23:15:24.446871932 (13h35m58.961974519s) [upload=13992]: Got next segment. Inline: true Aug 7 23:15:24.446878746 (13h35m58.961981315s) [upload=13992]: Waiting for error group managing segments... Aug 7 23:15:24.446939653 (13h35m58.962042222s) [upload=13992]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:24.446945288 (13h35m58.962047863s) [upload=13992]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:24.446950467 (13h35m58.96205303s) [upload=13992]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:54.414392139 (13h36m28.929494705s) [upload=13992]: Done uploading segments... Aug 7 23:15:54.414412085 (13h36m28.929514651s) [upload=13992]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:32.150263694 (13h37m6.665366261s) [upload=14009]: Starting upload Aug 7 23:16:32.150345034 (13h37m6.665447612s) [upload=14009]: Done starting upload Aug 7 23:16:32.150402507 (13h37m6.665505071s) [upload=14009]: Uploading segments... Aug 7 23:16:32.150453597 (13h37m6.665556176s) [upload=14009]: Got next segment. Inline: true Aug 7 23:16:32.150458385 (13h37m6.665560958s) [upload=14009]: Waiting for error group managing segments... Aug 7 23:16:32.150501065 (13h37m6.665603641s) [upload=14009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:32.150505833 (13h37m6.66560841s) [upload=14009]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:32.150509597 (13h37m6.665612174s) [upload=14009]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:02.044102103 (13h37m36.55920467s) [upload=14009]: Done uploading segments... Aug 7 23:17:02.044123177 (13h37m36.559225743s) [upload=14009]: Upload finished. err: metaclient: context deadline exceeded Duration: 243.324066ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......................XX.XXXX...X.X........................] duration: 782.744556ms FAIL (2025-08-07 23:40:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ce_id=GH7IOGMAIMB7DEPPOCFSWD2JIW2T3GGESTVFX4YHDDODF2UKVV4A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.270963124 (13h41m24.786065727s) [noise=true , address=104.36.82.61:4398 , piece_id=6JGM2OSGIKGQJVSGKTYSKXWFWUPD2CZBJE5SUVHZTHIDXKYWSVFQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271303186 (13h41m24.786405766s) [noise=true , address=45.91.116.14:4198 , piece_id=PPA53436A4JFKNVU3PBE7WSTEWHVNITSNMU2VYCZU5DDZCFJOPSQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271556719 (13h41m24.786659309s) [noise=true , address=104.36.82.9:4034 , piece_id=UDZJLVKKAFC3IJGML35BAWKAYPYKCAGODNJBAK4R6QKEQDS5EZSA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271787526 (13h41m24.786890105s) [noise=true , address=104.36.82.42:4808 , piece_id=G6LVWECALGP45ILRGEK3X3UHOOQHP2UREVWFCDPYJZC3DFMPNYMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.272659277 (13h41m24.787761864s) [noise=true , address=64.49.13.27:4967 , piece_id=53RPNRVKZ6TYT7VP2DCXAX33YMUMDAWZS3RMRS53AW3OSEDIPCTA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.272784283 (13h41m24.787886875s) [noise=true , address=104.36.82.16:4326 , piece_id=DQUUJR7LJFQJQUKRTWXHS5SCMU4ZZ72EIYZZJY25HRM2BJYHLR7A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.273228084 (13h41m24.788330665s) [noise=true , address=104.36.82.60:4353 , piece_id=T26XKES2P2UK4QHLVWKJUZNS2IXZUYWC7FU45LGU7SC2SJILXFMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.273443857 (13h41m24.788546457s) [noise=true , address=104.36.82.44:4900 , piece_id=PS3WKIHAI4T25TOTOGXJH5RMXCCKYJQJ5JAX52ODW2G24VXLZUSQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.274091759 (13h41m24.789194346s) [noise=true , address=104.36.82.2:4759 , piece_id=WQ42DKSHG5FXUCDKII7BNEFDA25UX6Q3ARG3DXTHV7OKIBM5NHEQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.275262302 (13h41m24.790364888s) [noise=true , address=79.127.220.97:4396 , piece_id=UQGXEYE4PKBTNQO2W73CX3W5IKMVDVDDSN3ZXYOWIJGTREKQF5VQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.275755725 (13h41m24.790858314s) [noise=true , address=104.36.82.56:4236 , piece_id=PTBEE4LERZ3JO3OPPJFAIBQ5LYSTEDZCVU2FSA7QZFZJUK7Q5TQA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.276176266 (13h41m24.791278854s) [noise=true , address=104.36.82.4:4811 , piece_id=T3ZIA6J7HELFDWFFQUTJK7LGIMXOKOV5ZGURFEIKGJRBU4YWW3TA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.276690914 (13h41m24.791793502s) [noise=true , address=45.91.116.10:4104 , piece_id=ZIUMAHVVTT6X7ZVWOJI2ELFYQG7QXFP2W4CTX2PWDSCSBIZU2TMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.277946066 (13h41m24.793048658s) [noise=true , address=156.146.43.225:4447 , piece_id=QMDN5UIK3XYXB2X35EVI33U2ANOEWIU5XUU2A7CFYRSRRJLDFW4Q, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.278411190 (13h41m24.793513776s) [noise=true , address=104.36.82.23:4463 , piece_id=AF7VUSU4PDWMINEEJ3CLAQ5GYANWSBTB7ZX3KK7BBPWKX575DY5A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.363417372 (13h41m24.878519955s) [upload=14080]: Next returned nil segment Aug 7 23:20:50.363434233 (13h41m24.878536817s) [upload=14080]: Waiting for error group managing segments... Aug 7 23:20:50.370437337 (13h41m24.885539907s) [noise=true , address=45.91.116.14:4198 , piece_id=PPA53436A4JFKNVU3PBE7WSTEWHVNITSNMU2VYCZU5DDZCFJOPSQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.371668429 (13h41m24.886771s) [noise=true , address=104.36.82.56:4236 , piece_id=PTBEE4LERZ3JO3OPPJFAIBQ5LYSTEDZCVU2FSA7QZFZJUK7Q5TQA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375301858 (13h41m24.890404435s) [noise=true , address=104.36.82.23:4463 , piece_id=AF7VUSU4PDWMINEEJ3CLAQ5GYANWSBTB7ZX3KK7BBPWKX575DY5A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375604014 (13h41m24.890706595s) [noise=true , address=104.36.82.42:4808 , piece_id=G6LVWECALGP45ILRGEK3X3UHOOQHP2UREVWFCDPYJZC3DFMPNYMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375817725 (13h41m24.89092029s) [noise=true , address=104.36.82.47:4535 , piece_id=XUCXMLKMK2JHA5RRZPMYTGC5Z6JB4BFI3J6BT5RM65O4KEEI3LEA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.376297840 (13h41m24.89140042s) [noise=true , address=104.36.82.4:4811 , piece_id=T3ZIA6J7HELFDWFFQUTJK7LGIMXOKOV5ZGURFEIKGJRBU4YWW3TA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.376376567 (13h41m24.89147913s) [noise=true , address=104.36.82.9:4034 , piece_id=UDZJLVKKAFC3IJGML35BAWKAYPYKCAGODNJBAK4R6QKEQDS5EZSA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.378451878 (13h41m24.893554445s) [noise=true , address=104.36.82.37:4673 , piece_id=GH7IOGMAIMB7DEPPOCFSWD2JIW2T3GGESTVFX4YHDDODF2UKVV4A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.387814655 (13h41m24.902917224s) [noise=true , address=104.36.82.2:4759 , piece_id=WQ42DKSHG5FXUCDKII7BNEFDA25UX6Q3ARG3DXTHV7OKIBM5NHEQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.393629027 (13h41m24.908731604s) [noise=true , address=45.91.116.10:4104 , piece_id=ZIUMAHVVTT6X7ZVWOJI2ELFYQG7QXFP2W4CTX2PWDSCSBIZU2TMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.399762700 (13h41m24.914865269s) [noise=true , address=104.36.82.60:4353 , piece_id=T26XKES2P2UK4QHLVWKJUZNS2IXZUYWC7FU45LGU7SC2SJILXFMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.461291683 (13h41m24.976394246s) [noise=true , address=79.127.220.97:4396 , piece_id=UQGXEYE4PKBTNQO2W73CX3W5IKMVDVDDSN3ZXYOWIJGTREKQF5VQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.478584042 (13h41m24.993686633s) [noise=true , address=104.36.82.61:4398 , piece_id=6JGM2OSGIKGQJVSGKTYSKXWFWUPD2CZBJE5SUVHZTHIDXKYWSVFQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.520830872 (13h41m25.035933449s) [noise=true , address=156.146.43.225:4447 , piece_id=QMDN5UIK3XYXB2X35EVI33U2ANOEWIU5XUU2A7CFYRSRRJLDFW4Q, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.520859322 (13h41m25.035961898s) [seg_pos={0 0}, upload=14080]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:50.521038126 (13h41m25.036140694s) [noise=true , address=104.36.82.44:4900 , piece_id=PS3WKIHAI4T25TOTOGXJH5RMXCCKYJQJ5JAX52ODW2G24VXLZUSQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1R8K65zsCspu7W2oAnvtpz7zEsQFkDG1e5BAp6faQYMQrkh4oQ): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:50.521126310 (13h41m25.03622888s) [noise=true , address=104.36.82.5:4842 , piece_id=AOQNSYWK7J4PGSTM5JEABQ34AFL33ZNKJRI77WO6GY5RKO7DAIHQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn): protocol: write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection; write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection Aug 7 23:20:50.521313827 (13h41m25.036416407s) [noise=true , address=104.36.82.16:4326 , piece_id=DQUUJR7LJFQJQUKRTWXHS5SCMU4ZZ72EIYZZJY25HRM2BJYHLR7A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pU8MZZKS88CPFdZoaQ4msNfkCbCewTmS2wKEdN7pG5aCWoAr6): protocol: read tcp 159.223.143.39:35084->104.36.82.16:4326: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:20:50.521375844 (13h41m25.03647841s) [noise=true , address=64.49.13.27:4967 , piece_id=53RPNRVKZ6TYT7VP2DCXAX33YMUMDAWZS3RMRS53AW3OSEDIPCTA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nVt2Ze4c5jQTyqFQDsJVaZKqge6gfjwpxUZXDCx8UVcySk4r8): protocol: read tcp 159.223.143.39:54110->64.49.13.27:4967: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:20:50.521411430 (13h41m25.036513997s) [seg_pos={0 0}, upload=14080]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:50.521520073 (13h41m25.03662264s) [upload=14080]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:50.521525409 (13h41m25.036627984s) [upload=14080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.573889476 (13h41m25.088992052s) [upload=14080]: Done uploading segments... Aug 7 23:20:50.573905889 (13h41m25.089008466s) [upload=14080]: Upload finished. err: <nil> Aug 7 23:21:30.905760263 (13h42m5.42086284s) [upload=14094]: Starting upload Aug 7 23:21:30.905840100 (13h42m5.420942676s) [upload=14094]: Done starting upload Aug 7 23:21:30.905958959 (13h42m5.421061527s) [upload=14094]: Uploading segments... Aug 7 23:21:30.906533404 (13h42m5.421636042s) [upload=14094]: Got next segment. Inline: false Aug 7 23:21:30.906593461 (13h42m5.421696028s) [upload=14094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:21:30.906598444 (13h42m5.42170102s) [upload=14094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:22:30.875535565 (13h43m5.390638142s) [upload=14094]: Next segment err: context deadline exceeded Aug 7 23:22:30.875554779 (13h43m5.390657361s) [upload=14094]: Done uploading segments... Aug 7 23:22:30.875570371 (13h43m5.390672939s) [upload=14094]: Upload finished. err: context deadline exceeded Duration: 782.792018ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 29.749968ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:40:41 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 29.771551ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................XXX..X.X.X........................] duration: 531.375188ms FAIL (2025-08-07 23:40:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 531.397929ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.....................XX.X.X....XX...........................] duration: 284.215738ms FAIL (2025-08-07 23:40:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 284.245724ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [......................XX..X...X.............................] duration: 247.262085ms FAIL (2025-08-07 23:40:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 162725354 (13h31m20.67782793s) [upload=13919]: Starting upload Aug 7 23:10:46.162807728 (13h31m20.677910304s) [upload=13919]: Done starting upload Aug 7 23:10:46.162877771 (13h31m20.677980347s) [upload=13919]: Uploading segments... Aug 7 23:10:46.162939319 (13h31m20.678041901s) [upload=13919]: Got next segment. Inline: true Aug 7 23:10:46.162944044 (13h31m20.678046624s) [upload=13919]: Waiting for error group managing segments... Aug 7 23:10:46.162986704 (13h31m20.678089286s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:46.162991049 (13h31m20.678093629s) [upload=13919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:46.162994213 (13h31m20.678096792s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:49.331481110 (13h31m23.84658369s) [upload=13919]: Done uploading segments... Aug 7 23:10:49.331502710 (13h31m23.846605285s) [upload=13919]: Upload finished. err: <nil> Aug 7 23:11:56.588799333 (13h32m31.103901909s) [upload=13942]: Starting upload Aug 7 23:11:56.588871172 (13h32m31.103973747s) [upload=13942]: Done starting upload Aug 7 23:11:56.588959193 (13h32m31.104061755s) [upload=13942]: Uploading segments... Aug 7 23:11:56.588996486 (13h32m31.104099072s) [upload=13942]: Got next segment. Inline: true Aug 7 23:11:56.589000914 (13h32m31.104103491s) [upload=13942]: Waiting for error group managing segments... Aug 7 23:11:56.589047870 (13h32m31.104150433s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:56.589051856 (13h32m31.104154432s) [upload=13942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:56.589054372 (13h32m31.10415695s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:56.555383755 (13h33m31.070486344s) [upload=13942]: Done uploading segments... Aug 7 23:12:56.555407795 (13h33m31.070510378s) [upload=13942]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:56.669023639 (13h33m31.184126201s) [upload=13959]: Starting upload Aug 7 23:12:56.669081512 (13h33m31.184184088s) [upload=13959]: Done starting upload Aug 7 23:12:56.669139265 (13h33m31.184241827s) [upload=13959]: Uploading segments... Aug 7 23:12:56.669181725 (13h33m31.184284307s) [upload=13959]: Got next segment. Inline: true Aug 7 23:12:56.669185727 (13h33m31.18428829s) [upload=13959]: Waiting for error group managing segments... Aug 7 23:12:56.669222629 (13h33m31.184325191s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:56.669226085 (13h33m31.18432866s) [upload=13959]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:56.669228556 (13h33m31.184331118s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.639927224 (13h34m31.155029791s) [upload=13959]: Done uploading segments... Aug 7 23:13:56.639946863 (13h34m31.155049429s) [upload=13959]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:56.754611797 (13h34m31.269714367s) [upload=13974]: Starting upload Aug 7 23:13:56.754690544 (13h34m31.269793123s) [upload=13974]: Done starting upload Aug 7 23:13:56.754787910 (13h34m31.269890478s) [upload=13974]: Uploading segments... Aug 7 23:13:56.754829279 (13h34m31.269931864s) [upload=13974]: Got next segment. Inline: true Aug 7 23:13:56.754833834 (13h34m31.269936411s) [upload=13974]: Waiting for error group managing segments... Aug 7 23:13:56.754887650 (13h34m31.269990217s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.754891993 (13h34m31.26999456s) [upload=13974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.754895495 (13h34m31.269998071s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.814228870 (13h34m31.32933145s) [upload=13974]: Done uploading segments... Aug 7 23:13:56.814253732 (13h34m31.329356295s) [upload=13974]: Upload finished. err: <nil> Aug 7 23:14:51.237083244 (13h35m25.75218581s) [upload=13984]: Starting upload Aug 7 23:14:51.237154440 (13h35m25.752257016s) [upload=13984]: Done starting upload Aug 7 23:14:51.237222602 (13h35m25.752325168s) [upload=13984]: Uploading segments... Aug 7 23:14:51.237272573 (13h35m25.752375158s) [upload=13984]: Got next segment. Inline: true Aug 7 23:14:51.237277088 (13h35m25.752379654s) [upload=13984]: Waiting for error group managing segments... Aug 7 23:14:51.237328372 (13h35m25.752430937s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:51.237332974 (13h35m25.75243555s) [upload=13984]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:51.237336117 (13h35m25.752438684s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:51.367347705 (13h35m25.882450281s) [upload=13984]: Done uploading segments... Aug 7 23:14:51.367368302 (13h35m25.882470865s) [upload=13984]: Upload finished. err: <nil> Aug 7 23:16:05.237199068 (13h36m39.752301655s) [upload=14002]: Starting upload Aug 7 23:16:05.237269888 (13h36m39.752372457s) [upload=14002]: Done starting upload Aug 7 23:16:05.237360382 (13h36m39.752462952s) [upload=14002]: Uploading segments... Aug 7 23:16:05.237841750 (13h36m39.752944336s) [upload=14002]: Got next segment. Inline: true Aug 7 23:16:05.237859632 (13h36m39.752962218s) [upload=14002]: Waiting for error group managing segments... Aug 7 23:16:05.237930004 (13h36m39.753032586s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:05.237936533 (13h36m39.753039109s) [upload=14002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:05.237940573 (13h36m39.753043148s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:05.205359206 (13h37m39.720461782s) [upload=14002]: Done uploading segments... Aug 7 23:17:05.205376470 (13h37m39.720479046s) [upload=14002]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:15.921193047 (13h37m50.436295623s) [upload=14020]: Starting upload Aug 7 23:17:15.921258731 (13h37m50.436361294s) [upload=14020]: Done starting upload Aug 7 23:17:15.921299906 (13h37m50.436402469s) [upload=14020]: Uploading segments... Aug 7 23:17:15.921331213 (13h37m50.436433799s) [upload=14020]: Got next segment. Inline: true Aug 7 23:17:15.921334685 (13h37m50.436437248s) [upload=14020]: Waiting for error group managing segments... Aug 7 23:17:15.921366597 (13h37m50.436469172s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:15.921370316 (13h37m50.436472878s) [upload=14020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:15.921372605 (13h37m50.436475167s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:15.996901858 (13h37m50.512004437s) [upload=14020]: Done uploading segments... Aug 7 23:17:15.996920869 (13h37m50.512023431s) [upload=14020]: Upload finished. err: <nil> Aug 7 23:18:10.846312036 (13h38m45.361414612s) [upload=14034]: Starting upload Aug 7 23:18:10.846377244 (13h38m45.36147981s) [upload=14034]: Done starting upload Aug 7 23:18:10.846461339 (13h38m45.361563904s) [upload=14034]: Uploading segments... Aug 7 23:18:10.846496473 (13h38m45.361599058s) [upload=14034]: Got next segment. Inline: true Aug 7 23:18:10.846501164 (13h38m45.36160374s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:18:10.846554786 (13h38m45.361657362s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:10.846559294 (13h38m45.361661871s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:10.846562922 (13h38m45.361665498s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:11.200018957 (13h38m45.715121534s) [upload=14034]: Done uploading segments... Aug 7 23:18:11.200049582 (13h38m45.715152158s) [upload=14034]: Upload finished. err: <nil> Aug 7 23:19:15.896167300 (13h39m50.411269878s) [upload=14051]: Starting upload Aug 7 23:19:15.896353995 (13h39m50.411456637s) [upload=14051]: Done starting upload Aug 7 23:19:15.896436268 (13h39m50.411538832s) [upload=14051]: Uploading segments... Aug 7 23:19:15.896501153 (13h39m50.411603732s) [upload=14051]: Got next segment. Inline: true Aug 7 23:19:15.896505978 (13h39m50.411608555s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:19:15.896570054 (13h39m50.411672631s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:15.896574888 (13h39m50.411677465s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:15.896578764 (13h39m50.411681341s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:15.976040109 (13h39m50.491142676s) [upload=14051]: Done uploading segments... Aug 7 23:19:15.976057062 (13h39m50.491159637s) [upload=14051]: Upload finished. err: <nil> Aug 7 23:19:50.253965481 (13h40m24.76906806s) [upload=14062]: Starting upload Aug 7 23:19:50.254050632 (13h40m24.769153219s) [upload=14062]: Done starting upload Aug 7 23:19:50.254136562 (13h40m24.769239133s) [upload=14062]: Uploading segments... Aug 7 23:19:50.254189310 (13h40m24.769291896s) [upload=14062]: Got next segment. Inline: true Aug 7 23:19:50.254194507 (13h40m24.769297084s) [upload=14062]: Waiting for error group managing segments... Aug 7 23:19:50.254255974 (13h40m24.769358555s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:50.254262645 (13h40m24.769365222s) [upload=14062]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:50.254267031 (13h40m24.769369607s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.224097554 (13h41m24.739200131s) [upload=14062]: Done uploading segments... Aug 7 23:20:50.224116508 (13h41m24.739219086s) [upload=14062]: Upload finished. err: metaclient: context deadline exceeded Duration: 247.288569ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 30.954579ms target: https://us1.storj.io/login FAIL (2025-08-07 23:40:24 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 30.97463ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................XXXXX................................] duration: 1.994585982s FAIL (2025-08-07 23:40:20 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.994613192s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [.......................X..XXXXX.X.XX........................] duration: 231.476806ms FAIL (2025-08-07 23:40:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 231.502768ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 533.016705ms FAIL (2025-08-07 23:40:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:11:04.814410393 (13h31m39.329512969s) [upload=13925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:11:04.814415016 (13h31m39.329517592s) [upload=13925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:04.947583472 (13h31m39.462686071s) [upload=13925]: Done uploading segments... Aug 7 23:11:04.947609736 (13h31m39.462712324s) [upload=13925]: Upload finished. err: <nil> Aug 7 23:11:29.227536921 (13h32m3.742639487s) [upload=13931]: Starting upload Aug 7 23:11:29.227600047 (13h32m3.742702623s) [upload=13931]: Done starting upload Aug 7 23:11:29.227715021 (13h32m3.742817591s) [upload=13931]: Uploading segments... Aug 7 23:11:29.227850209 (13h32m3.74295278s) [upload=13931]: Got next segment. Inline: false Aug 7 23:11:29.227898592 (13h32m3.743001169s) [upload=13931]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:29.227903365 (13h32m3.743005942s) [upload=13931]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:11:47.514990540 (13h32m22.03009312s) [seg_pos={0 0}, upload=13931]: Begin upload segment... Aug 7 23:11:47.515067094 (13h32m22.030169657s) [seg_pos={0 0}, upload=13931]: Done begin upload segment. Aug 7 23:11:47.515129228 (13h32m22.030231814s) [noise=true , address=77.90.31.18:4806 , piece_id=LI5HUM3HCYFYYSO6BENU4DQ2SFOPEJLRWZRM5PWCFXUZON4WCMAA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.515554241 (13h32m22.03065682s) [noise=true , address=104.36.82.14:4222 , piece_id=B5WB353FQ47THJP7MXV4AY5Z6FFSG7A66X72BLJUDK2ZTM3FNDVA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.516068554 (13h32m22.031171143s) [noise=true , address=45.91.116.7:4010 , piece_id=XOROULN3EEUSG45LZ5V53ORP2YV7CWLRQDSQ2AXREAFB62FGNL6A, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.516625763 (13h32m22.031728346s) [noise=true , address=45.91.116.8:4047 , piece_id=5E2VVSY65OAD6OFCQBBOTYLCLUKNVCVD3KY7OOVBI5TCYKQUKBQQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517060179 (13h32m22.03216276s) [noise=true , address=104.36.82.7:4910 , piece_id=Y75KUQR2IYRZBANBQPDFZTUJZPM54BA3KI6ZEF77NI3V755HPBOA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517396640 (13h32m22.032499217s) [noise=true , address=104.36.82.6:4873 , piece_id=3YGEWHQ7MXU6IPUJCYNM7L4DMNOECDSTVXMUBCX3CJCGPJ4OGGIQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517801490 (13h32m22.032904072s) [noise=true , address=45.91.116.10:4081 , piece_id=WDJ6KCU4LP75N7TSSR2VTWWEON3VRWE7BFJL57H2RM7ZYI2GYLWQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518135215 (13h32m22.033237797s) [noise=true , address=45.91.116.5:4297 , piece_id=T76WY44JQ55TJH4KJM57QWNBP7DVYNIBDVAPVN2AUTWASN4V76IA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518441967 (13h32m22.033544543s) [noise=true , address=77.90.31.11:4553 , piece_id=NK6AJDEDIAA5N3DNKBIPKZXURRSBOJG26FIMUB2SVFTIWOKFJWNQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518756911 (13h32m22.033859486s) [noise=true , address=77.90.31.21:4877 , piece_id=NVD7UBISRZJFOBJR7JKOX6V354Z6VCEP5TWJPGW3WOSI6S6K5FDA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519217404 (13h32m22.034319986s) [noise=true , address=45.91.116.6:4973 , piece_id=HEA5GQ53KZVIMBZ5LC3S4Y4372WMGUSATCBDTY5BIHZ4WL2PG7HQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519516685 (13h32m22.034619271s) [noise=true , address=77.90.31.19:4841 , piece_id=Z76CMQDB3DDLHQV35DZORZWRBX2D52BLBDCKPFXPONE3OUNGSGHA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519904148 (13h32m22.035006727s) [noise=true , address=77.90.31.19:4842 , piece_id=Q7PUVWG4BXEJSS4YOBVTMUHCAWWKKQAR43NH4U3R24YMK3MP6H6Q, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.520220651 (13h32m22.035323239s) [noise=true , address=77.90.31.10:4517 , piece_id=WJ7TMWHMURKZ6HJON2NBIZVIC4XEXUUR5M7KD5PFWV4MEB3DUV7A, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.520644038 (13h32m22.035746619s) [noise=true , address=104.36.82.12:4149 , piece_id=UEFYG73QPD3OQX5IF2BGE7SQHYL5OKV37LQOKGNECBGRYQLULCJQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.521219880 (13h32m22.036322466s) [noise=true , address=77.90.31.17:4771 , piece_id=YCZ26U3QI4NQ7CKNVZEEB4O6XK4ZR3SPXTKT75P5U4ZQLUWOJWQA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.609483246 (13h32m22.124585826s) [upload=13931]: Next returned nil segment Aug 7 23:11:47.609501768 (13h32m22.124604348s) [upload=13931]: Waiting for error group managing segments... Aug 7 23:11:47.614240475 (13h32m22.129343046s) [noise=true , address=45.91.116.8:4047 , piece_id=5E2VVSY65OAD6OFCQBBOTYLCLUKNVCVD3KY7OOVBI5TCYKQUKBQQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.614925369 (13h32m22.130027939s) [noise=true , address=45.91.116.5:4297 , piece_id=T76WY44JQ55TJH4KJM57QWNBP7DVYNIBDVAPVN2AUTWASN4V76IA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.615059457 (13h32m22.130162026s) [noise=true , address=45.91.116.7:4010 , piece_id=XOROULN3EEUSG45LZ5V53ORP2YV7CWLRQDSQ2AXREAFB62FGNL6A, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.615458775 (13h32m22.130561338s) [noise=true , address=77.90.31.19:4842 , piece_id=Q7PUVWG4BXEJSS4YOBVTMUHCAWWKKQAR43NH4U3R24YMK3MP6H6Q, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.616116276 (13h32m22.131218838s) [noise=true , address=77.90.31.18:4806 , piece_id=LI5HUM3HCYFYYSO6BENU4DQ2SFOPEJLRWZRM5PWCFXUZON4WCMAA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.616661348 (13h32m22.131763909s) [noise=true , address=77.90.31.17:4771 , piece_id=YCZ26U3QI4NQ7CKNVZEEB4O6XK4ZR3SPXTKT75P5U4ZQLUWOJWQA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.618000416 (13h32m22.133102978s) [noise=true , address=45.91.116.10:4081 , piece_id=WDJ6KCU4LP75N7TSSR2VTWWEON3VRWE7BFJL57H2RM7ZYI2GYLWQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.618530300 (13h32m22.133632866s) [noise=true , address=45.91.116.6:4973 , piece_id=HEA5GQ53KZVIMBZ5LC3S4Y4372WMGUSATCBDTY5BIHZ4WL2PG7HQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.619098772 (13h32m22.13420134s) [noise=true , address=77.90.31.19:4841 , piece_id=Z76CMQDB3DDLHQV35DZORZWRBX2D52BLBDCKPFXPONE3OUNGSGHA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.619697172 (13h32m22.13479975s) [noise=true , address=77.90.31.11:4553 , piece_id=NK6AJDEDIAA5N3DNKBIPKZXURRSBOJG26FIMUB2SVFTIWOKFJWNQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.620954558 (13h32m22.136057128s) [noise=true , address=77.90.31.21:4877 , piece_id=NVD7UBISRZJFOBJR7JKOX6V354Z6VCEP5TWJPGW3WOSI6S6K5FDA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.627010966 (13h32m22.142113543s) [noise=true , address=104.36.82.7:4910 , piece_id=Y75KUQR2IYRZBANBQPDFZTUJZPM54BA3KI6ZEF77NI3V755HPBOA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.629783113 (13h32m22.144885677s) [noise=true , address=77.90.31.10:4517 , piece_id=WJ7TMWHMURKZ6HJON2NBIZVIC4XEXUUR5M7KD5PFWV4MEB3DUV7A, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.629816031 (13h32m22.144918609s) [seg_pos={0 0}, upload=13931]: Segment reached optimal threshold of 13 pieces. Aug 7 23:11:47.630126623 (13h32m22.145229201s) [noise=true , address=104.36.82.14:4222 , piece_id=B5WB353FQ47THJP7MXV4AY5Z6FFSG7A66X72BLJUDK2ZTM3FNDVA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection; write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection Aug 7 23:11:47.630137290 (13h32m22.145239874s) [noise=true , address=104.36.82.12:4149 , piece_id=UEFYG73QPD3OQX5IF2BGE7SQHYL5OKV37LQOKGNECBGRYQLULCJQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection; write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection Aug 7 23:11:47.630232787 (13h32m22.145335361s) [noise=true , address=104.36.82.6:4873 , piece_id=3YGEWHQ7MXU6IPUJCYNM7L4DMNOECDSTVXMUBCX3CJCGPJ4OGGIQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: read tcp 159.223.143.39:36062->104.36.82.6:4873: use of closed network connection; context canceled Aug 7 23:11:47.630264689 (13h32m22.145367273s) [seg_pos={0 0}, upload=13931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:11:47.630345104 (13h32m22.145447682s) [upload=13931]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:11:47.630350066 (13h32m22.145452652s) [upload=13931]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:37.334152859 (13h39m11.849255428s) [upload=13931]: Done uploading segments... Aug 7 23:18:37.334173589 (13h39m11.849276166s) [upload=13931]: Upload finished. err: metaclient: context deadline exceeded Duration: 533.053063ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......................XX..X...X............................] duration: 255.842671ms FAIL (2025-08-07 23:40:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 162725354 (13h31m20.67782793s) [upload=13919]: Starting upload Aug 7 23:10:46.162807728 (13h31m20.677910304s) [upload=13919]: Done starting upload Aug 7 23:10:46.162877771 (13h31m20.677980347s) [upload=13919]: Uploading segments... Aug 7 23:10:46.162939319 (13h31m20.678041901s) [upload=13919]: Got next segment. Inline: true Aug 7 23:10:46.162944044 (13h31m20.678046624s) [upload=13919]: Waiting for error group managing segments... Aug 7 23:10:46.162986704 (13h31m20.678089286s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:46.162991049 (13h31m20.678093629s) [upload=13919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:46.162994213 (13h31m20.678096792s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:49.331481110 (13h31m23.84658369s) [upload=13919]: Done uploading segments... Aug 7 23:10:49.331502710 (13h31m23.846605285s) [upload=13919]: Upload finished. err: <nil> Aug 7 23:11:56.588799333 (13h32m31.103901909s) [upload=13942]: Starting upload Aug 7 23:11:56.588871172 (13h32m31.103973747s) [upload=13942]: Done starting upload Aug 7 23:11:56.588959193 (13h32m31.104061755s) [upload=13942]: Uploading segments... Aug 7 23:11:56.588996486 (13h32m31.104099072s) [upload=13942]: Got next segment. Inline: true Aug 7 23:11:56.589000914 (13h32m31.104103491s) [upload=13942]: Waiting for error group managing segments... Aug 7 23:11:56.589047870 (13h32m31.104150433s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:56.589051856 (13h32m31.104154432s) [upload=13942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:56.589054372 (13h32m31.10415695s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:56.555383755 (13h33m31.070486344s) [upload=13942]: Done uploading segments... Aug 7 23:12:56.555407795 (13h33m31.070510378s) [upload=13942]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:56.669023639 (13h33m31.184126201s) [upload=13959]: Starting upload Aug 7 23:12:56.669081512 (13h33m31.184184088s) [upload=13959]: Done starting upload Aug 7 23:12:56.669139265 (13h33m31.184241827s) [upload=13959]: Uploading segments... Aug 7 23:12:56.669181725 (13h33m31.184284307s) [upload=13959]: Got next segment. Inline: true Aug 7 23:12:56.669185727 (13h33m31.18428829s) [upload=13959]: Waiting for error group managing segments... Aug 7 23:12:56.669222629 (13h33m31.184325191s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:56.669226085 (13h33m31.18432866s) [upload=13959]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:56.669228556 (13h33m31.184331118s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.639927224 (13h34m31.155029791s) [upload=13959]: Done uploading segments... Aug 7 23:13:56.639946863 (13h34m31.155049429s) [upload=13959]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:56.754611797 (13h34m31.269714367s) [upload=13974]: Starting upload Aug 7 23:13:56.754690544 (13h34m31.269793123s) [upload=13974]: Done starting upload Aug 7 23:13:56.754787910 (13h34m31.269890478s) [upload=13974]: Uploading segments... Aug 7 23:13:56.754829279 (13h34m31.269931864s) [upload=13974]: Got next segment. Inline: true Aug 7 23:13:56.754833834 (13h34m31.269936411s) [upload=13974]: Waiting for error group managing segments... Aug 7 23:13:56.754887650 (13h34m31.269990217s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.754891993 (13h34m31.26999456s) [upload=13974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.754895495 (13h34m31.269998071s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.814228870 (13h34m31.32933145s) [upload=13974]: Done uploading segments... Aug 7 23:13:56.814253732 (13h34m31.329356295s) [upload=13974]: Upload finished. err: <nil> Aug 7 23:14:51.237083244 (13h35m25.75218581s) [upload=13984]: Starting upload Aug 7 23:14:51.237154440 (13h35m25.752257016s) [upload=13984]: Done starting upload Aug 7 23:14:51.237222602 (13h35m25.752325168s) [upload=13984]: Uploading segments... Aug 7 23:14:51.237272573 (13h35m25.752375158s) [upload=13984]: Got next segment. Inline: true Aug 7 23:14:51.237277088 (13h35m25.752379654s) [upload=13984]: Waiting for error group managing segments... Aug 7 23:14:51.237328372 (13h35m25.752430937s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:51.237332974 (13h35m25.75243555s) [upload=13984]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:51.237336117 (13h35m25.752438684s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:51.367347705 (13h35m25.882450281s) [upload=13984]: Done uploading segments... Aug 7 23:14:51.367368302 (13h35m25.882470865s) [upload=13984]: Upload finished. err: <nil> Aug 7 23:16:05.237199068 (13h36m39.752301655s) [upload=14002]: Starting upload Aug 7 23:16:05.237269888 (13h36m39.752372457s) [upload=14002]: Done starting upload Aug 7 23:16:05.237360382 (13h36m39.752462952s) [upload=14002]: Uploading segments... Aug 7 23:16:05.237841750 (13h36m39.752944336s) [upload=14002]: Got next segment. Inline: true Aug 7 23:16:05.237859632 (13h36m39.752962218s) [upload=14002]: Waiting for error group managing segments... Aug 7 23:16:05.237930004 (13h36m39.753032586s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:05.237936533 (13h36m39.753039109s) [upload=14002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:05.237940573 (13h36m39.753043148s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:05.205359206 (13h37m39.720461782s) [upload=14002]: Done uploading segments... Aug 7 23:17:05.205376470 (13h37m39.720479046s) [upload=14002]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:15.921193047 (13h37m50.436295623s) [upload=14020]: Starting upload Aug 7 23:17:15.921258731 (13h37m50.436361294s) [upload=14020]: Done starting upload Aug 7 23:17:15.921299906 (13h37m50.436402469s) [upload=14020]: Uploading segments... Aug 7 23:17:15.921331213 (13h37m50.436433799s) [upload=14020]: Got next segment. Inline: true Aug 7 23:17:15.921334685 (13h37m50.436437248s) [upload=14020]: Waiting for error group managing segments... Aug 7 23:17:15.921366597 (13h37m50.436469172s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:15.921370316 (13h37m50.436472878s) [upload=14020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:15.921372605 (13h37m50.436475167s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:15.996901858 (13h37m50.512004437s) [upload=14020]: Done uploading segments... Aug 7 23:17:15.996920869 (13h37m50.512023431s) [upload=14020]: Upload finished. err: <nil> Aug 7 23:18:10.846312036 (13h38m45.361414612s) [upload=14034]: Starting upload Aug 7 23:18:10.846377244 (13h38m45.36147981s) [upload=14034]: Done starting upload Aug 7 23:18:10.846461339 (13h38m45.361563904s) [upload=14034]: Uploading segments... Aug 7 23:18:10.846496473 (13h38m45.361599058s) [upload=14034]: Got next segment. Inline: true Aug 7 23:18:10.846501164 (13h38m45.36160374s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:18:10.846554786 (13h38m45.361657362s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:10.846559294 (13h38m45.361661871s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:10.846562922 (13h38m45.361665498s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:11.200018957 (13h38m45.715121534s) [upload=14034]: Done uploading segments... Aug 7 23:18:11.200049582 (13h38m45.715152158s) [upload=14034]: Upload finished. err: <nil> Aug 7 23:19:15.896167300 (13h39m50.411269878s) [upload=14051]: Starting upload Aug 7 23:19:15.896353995 (13h39m50.411456637s) [upload=14051]: Done starting upload Aug 7 23:19:15.896436268 (13h39m50.411538832s) [upload=14051]: Uploading segments... Aug 7 23:19:15.896501153 (13h39m50.411603732s) [upload=14051]: Got next segment. Inline: true Aug 7 23:19:15.896505978 (13h39m50.411608555s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:19:15.896570054 (13h39m50.411672631s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:15.896574888 (13h39m50.411677465s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:15.896578764 (13h39m50.411681341s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:15.976040109 (13h39m50.491142676s) [upload=14051]: Done uploading segments... Aug 7 23:19:15.976057062 (13h39m50.491159637s) [upload=14051]: Upload finished. err: <nil> Aug 7 23:19:50.253965481 (13h40m24.76906806s) [upload=14062]: Starting upload Aug 7 23:19:50.254050632 (13h40m24.769153219s) [upload=14062]: Done starting upload Aug 7 23:19:50.254136562 (13h40m24.769239133s) [upload=14062]: Uploading segments... Aug 7 23:19:50.254189310 (13h40m24.769291896s) [upload=14062]: Got next segment. Inline: true Aug 7 23:19:50.254194507 (13h40m24.769297084s) [upload=14062]: Waiting for error group managing segments... Aug 7 23:19:50.254255974 (13h40m24.769358555s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:50.254262645 (13h40m24.769365222s) [upload=14062]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:50.254267031 (13h40m24.769369607s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.224097554 (13h41m24.739200131s) [upload=14062]: Done uploading segments... Aug 7 23:20:50.224116508 (13h41m24.739219086s) [upload=14062]: Upload finished. err: metaclient: context deadline exceeded Duration: 255.883318ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 58.11481ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:40:11 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 58.143732ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.....................X.XXX.XX...............................] duration: 835.640941ms FAIL (2025-08-07 23:40:10 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 835.674185ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [......................XX.X.X....XX..........................] duration: 341.352147ms FAIL (2025-08-07 23:40:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 342.266719ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.......................XX.X.X....XX.........................] duration: 253.204829ms FAIL (2025-08-07 23:39:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 253.224123ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [......................X.XXX.XX..............................] duration: 1.470648648s FAIL (2025-08-07 23:39:50 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.470686866s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [........................XX..X...X...........................] duration: 207.744253ms FAIL (2025-08-07 23:39:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 162725354 (13h31m20.67782793s) [upload=13919]: Starting upload Aug 7 23:10:46.162807728 (13h31m20.677910304s) [upload=13919]: Done starting upload Aug 7 23:10:46.162877771 (13h31m20.677980347s) [upload=13919]: Uploading segments... Aug 7 23:10:46.162939319 (13h31m20.678041901s) [upload=13919]: Got next segment. Inline: true Aug 7 23:10:46.162944044 (13h31m20.678046624s) [upload=13919]: Waiting for error group managing segments... Aug 7 23:10:46.162986704 (13h31m20.678089286s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:46.162991049 (13h31m20.678093629s) [upload=13919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:46.162994213 (13h31m20.678096792s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:49.331481110 (13h31m23.84658369s) [upload=13919]: Done uploading segments... Aug 7 23:10:49.331502710 (13h31m23.846605285s) [upload=13919]: Upload finished. err: <nil> Aug 7 23:11:56.588799333 (13h32m31.103901909s) [upload=13942]: Starting upload Aug 7 23:11:56.588871172 (13h32m31.103973747s) [upload=13942]: Done starting upload Aug 7 23:11:56.588959193 (13h32m31.104061755s) [upload=13942]: Uploading segments... Aug 7 23:11:56.588996486 (13h32m31.104099072s) [upload=13942]: Got next segment. Inline: true Aug 7 23:11:56.589000914 (13h32m31.104103491s) [upload=13942]: Waiting for error group managing segments... Aug 7 23:11:56.589047870 (13h32m31.104150433s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:56.589051856 (13h32m31.104154432s) [upload=13942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:56.589054372 (13h32m31.10415695s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:56.555383755 (13h33m31.070486344s) [upload=13942]: Done uploading segments... Aug 7 23:12:56.555407795 (13h33m31.070510378s) [upload=13942]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:56.669023639 (13h33m31.184126201s) [upload=13959]: Starting upload Aug 7 23:12:56.669081512 (13h33m31.184184088s) [upload=13959]: Done starting upload Aug 7 23:12:56.669139265 (13h33m31.184241827s) [upload=13959]: Uploading segments... Aug 7 23:12:56.669181725 (13h33m31.184284307s) [upload=13959]: Got next segment. Inline: true Aug 7 23:12:56.669185727 (13h33m31.18428829s) [upload=13959]: Waiting for error group managing segments... Aug 7 23:12:56.669222629 (13h33m31.184325191s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:56.669226085 (13h33m31.18432866s) [upload=13959]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:56.669228556 (13h33m31.184331118s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.639927224 (13h34m31.155029791s) [upload=13959]: Done uploading segments... Aug 7 23:13:56.639946863 (13h34m31.155049429s) [upload=13959]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:56.754611797 (13h34m31.269714367s) [upload=13974]: Starting upload Aug 7 23:13:56.754690544 (13h34m31.269793123s) [upload=13974]: Done starting upload Aug 7 23:13:56.754787910 (13h34m31.269890478s) [upload=13974]: Uploading segments... Aug 7 23:13:56.754829279 (13h34m31.269931864s) [upload=13974]: Got next segment. Inline: true Aug 7 23:13:56.754833834 (13h34m31.269936411s) [upload=13974]: Waiting for error group managing segments... Aug 7 23:13:56.754887650 (13h34m31.269990217s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.754891993 (13h34m31.26999456s) [upload=13974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.754895495 (13h34m31.269998071s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.814228870 (13h34m31.32933145s) [upload=13974]: Done uploading segments... Aug 7 23:13:56.814253732 (13h34m31.329356295s) [upload=13974]: Upload finished. err: <nil> Aug 7 23:14:51.237083244 (13h35m25.75218581s) [upload=13984]: Starting upload Aug 7 23:14:51.237154440 (13h35m25.752257016s) [upload=13984]: Done starting upload Aug 7 23:14:51.237222602 (13h35m25.752325168s) [upload=13984]: Uploading segments... Aug 7 23:14:51.237272573 (13h35m25.752375158s) [upload=13984]: Got next segment. Inline: true Aug 7 23:14:51.237277088 (13h35m25.752379654s) [upload=13984]: Waiting for error group managing segments... Aug 7 23:14:51.237328372 (13h35m25.752430937s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:51.237332974 (13h35m25.75243555s) [upload=13984]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:51.237336117 (13h35m25.752438684s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:51.367347705 (13h35m25.882450281s) [upload=13984]: Done uploading segments... Aug 7 23:14:51.367368302 (13h35m25.882470865s) [upload=13984]: Upload finished. err: <nil> Aug 7 23:16:05.237199068 (13h36m39.752301655s) [upload=14002]: Starting upload Aug 7 23:16:05.237269888 (13h36m39.752372457s) [upload=14002]: Done starting upload Aug 7 23:16:05.237360382 (13h36m39.752462952s) [upload=14002]: Uploading segments... Aug 7 23:16:05.237841750 (13h36m39.752944336s) [upload=14002]: Got next segment. Inline: true Aug 7 23:16:05.237859632 (13h36m39.752962218s) [upload=14002]: Waiting for error group managing segments... Aug 7 23:16:05.237930004 (13h36m39.753032586s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:05.237936533 (13h36m39.753039109s) [upload=14002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:05.237940573 (13h36m39.753043148s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:05.205359206 (13h37m39.720461782s) [upload=14002]: Done uploading segments... Aug 7 23:17:05.205376470 (13h37m39.720479046s) [upload=14002]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:15.921193047 (13h37m50.436295623s) [upload=14020]: Starting upload Aug 7 23:17:15.921258731 (13h37m50.436361294s) [upload=14020]: Done starting upload Aug 7 23:17:15.921299906 (13h37m50.436402469s) [upload=14020]: Uploading segments... Aug 7 23:17:15.921331213 (13h37m50.436433799s) [upload=14020]: Got next segment. Inline: true Aug 7 23:17:15.921334685 (13h37m50.436437248s) [upload=14020]: Waiting for error group managing segments... Aug 7 23:17:15.921366597 (13h37m50.436469172s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:15.921370316 (13h37m50.436472878s) [upload=14020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:15.921372605 (13h37m50.436475167s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:15.996901858 (13h37m50.512004437s) [upload=14020]: Done uploading segments... Aug 7 23:17:15.996920869 (13h37m50.512023431s) [upload=14020]: Upload finished. err: <nil> Aug 7 23:18:10.846312036 (13h38m45.361414612s) [upload=14034]: Starting upload Aug 7 23:18:10.846377244 (13h38m45.36147981s) [upload=14034]: Done starting upload Aug 7 23:18:10.846461339 (13h38m45.361563904s) [upload=14034]: Uploading segments... Aug 7 23:18:10.846496473 (13h38m45.361599058s) [upload=14034]: Got next segment. Inline: true Aug 7 23:18:10.846501164 (13h38m45.36160374s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:18:10.846554786 (13h38m45.361657362s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:10.846559294 (13h38m45.361661871s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:10.846562922 (13h38m45.361665498s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:11.200018957 (13h38m45.715121534s) [upload=14034]: Done uploading segments... Aug 7 23:18:11.200049582 (13h38m45.715152158s) [upload=14034]: Upload finished. err: <nil> Aug 7 23:19:15.896167300 (13h39m50.411269878s) [upload=14051]: Starting upload Aug 7 23:19:15.896353995 (13h39m50.411456637s) [upload=14051]: Done starting upload Aug 7 23:19:15.896436268 (13h39m50.411538832s) [upload=14051]: Uploading segments... Aug 7 23:19:15.896501153 (13h39m50.411603732s) [upload=14051]: Got next segment. Inline: true Aug 7 23:19:15.896505978 (13h39m50.411608555s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:19:15.896570054 (13h39m50.411672631s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:15.896574888 (13h39m50.411677465s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:15.896578764 (13h39m50.411681341s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:15.976040109 (13h39m50.491142676s) [upload=14051]: Done uploading segments... Aug 7 23:19:15.976057062 (13h39m50.491159637s) [upload=14051]: Upload finished. err: <nil> Aug 7 23:19:50.253965481 (13h40m24.76906806s) [upload=14062]: Starting upload Aug 7 23:19:50.254050632 (13h40m24.769153219s) [upload=14062]: Done starting upload Aug 7 23:19:50.254136562 (13h40m24.769239133s) [upload=14062]: Uploading segments... Aug 7 23:19:50.254189310 (13h40m24.769291896s) [upload=14062]: Got next segment. Inline: true Aug 7 23:19:50.254194507 (13h40m24.769297084s) [upload=14062]: Waiting for error group managing segments... Aug 7 23:19:50.254255974 (13h40m24.769358555s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:50.254262645 (13h40m24.769365222s) [upload=14062]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:50.254267031 (13h40m24.769369607s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.224097554 (13h41m24.739200131s) [upload=14062]: Done uploading segments... Aug 7 23:20:50.224116508 (13h41m24.739219086s) [upload=14062]: Upload finished. err: metaclient: context deadline exceeded Duration: 207.885319ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 28.462862ms target: https://us1.storj.io/login FAIL (2025-08-07 23:39:46 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 28.506237ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 593.776874ms FAIL (2025-08-07 23:39:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:11:04.814410393 (13h31m39.329512969s) [upload=13925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:11:04.814415016 (13h31m39.329517592s) [upload=13925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:04.947583472 (13h31m39.462686071s) [upload=13925]: Done uploading segments... Aug 7 23:11:04.947609736 (13h31m39.462712324s) [upload=13925]: Upload finished. err: <nil> Aug 7 23:11:29.227536921 (13h32m3.742639487s) [upload=13931]: Starting upload Aug 7 23:11:29.227600047 (13h32m3.742702623s) [upload=13931]: Done starting upload Aug 7 23:11:29.227715021 (13h32m3.742817591s) [upload=13931]: Uploading segments... Aug 7 23:11:29.227850209 (13h32m3.74295278s) [upload=13931]: Got next segment. Inline: false Aug 7 23:11:29.227898592 (13h32m3.743001169s) [upload=13931]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:29.227903365 (13h32m3.743005942s) [upload=13931]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:11:47.514990540 (13h32m22.03009312s) [seg_pos={0 0}, upload=13931]: Begin upload segment... Aug 7 23:11:47.515067094 (13h32m22.030169657s) [seg_pos={0 0}, upload=13931]: Done begin upload segment. Aug 7 23:11:47.515129228 (13h32m22.030231814s) [noise=true , address=77.90.31.18:4806 , piece_id=LI5HUM3HCYFYYSO6BENU4DQ2SFOPEJLRWZRM5PWCFXUZON4WCMAA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.515554241 (13h32m22.03065682s) [noise=true , address=104.36.82.14:4222 , piece_id=B5WB353FQ47THJP7MXV4AY5Z6FFSG7A66X72BLJUDK2ZTM3FNDVA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.516068554 (13h32m22.031171143s) [noise=true , address=45.91.116.7:4010 , piece_id=XOROULN3EEUSG45LZ5V53ORP2YV7CWLRQDSQ2AXREAFB62FGNL6A, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.516625763 (13h32m22.031728346s) [noise=true , address=45.91.116.8:4047 , piece_id=5E2VVSY65OAD6OFCQBBOTYLCLUKNVCVD3KY7OOVBI5TCYKQUKBQQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517060179 (13h32m22.03216276s) [noise=true , address=104.36.82.7:4910 , piece_id=Y75KUQR2IYRZBANBQPDFZTUJZPM54BA3KI6ZEF77NI3V755HPBOA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517396640 (13h32m22.032499217s) [noise=true , address=104.36.82.6:4873 , piece_id=3YGEWHQ7MXU6IPUJCYNM7L4DMNOECDSTVXMUBCX3CJCGPJ4OGGIQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517801490 (13h32m22.032904072s) [noise=true , address=45.91.116.10:4081 , piece_id=WDJ6KCU4LP75N7TSSR2VTWWEON3VRWE7BFJL57H2RM7ZYI2GYLWQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518135215 (13h32m22.033237797s) [noise=true , address=45.91.116.5:4297 , piece_id=T76WY44JQ55TJH4KJM57QWNBP7DVYNIBDVAPVN2AUTWASN4V76IA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518441967 (13h32m22.033544543s) [noise=true , address=77.90.31.11:4553 , piece_id=NK6AJDEDIAA5N3DNKBIPKZXURRSBOJG26FIMUB2SVFTIWOKFJWNQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518756911 (13h32m22.033859486s) [noise=true , address=77.90.31.21:4877 , piece_id=NVD7UBISRZJFOBJR7JKOX6V354Z6VCEP5TWJPGW3WOSI6S6K5FDA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519217404 (13h32m22.034319986s) [noise=true , address=45.91.116.6:4973 , piece_id=HEA5GQ53KZVIMBZ5LC3S4Y4372WMGUSATCBDTY5BIHZ4WL2PG7HQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519516685 (13h32m22.034619271s) [noise=true , address=77.90.31.19:4841 , piece_id=Z76CMQDB3DDLHQV35DZORZWRBX2D52BLBDCKPFXPONE3OUNGSGHA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519904148 (13h32m22.035006727s) [noise=true , address=77.90.31.19:4842 , piece_id=Q7PUVWG4BXEJSS4YOBVTMUHCAWWKKQAR43NH4U3R24YMK3MP6H6Q, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.520220651 (13h32m22.035323239s) [noise=true , address=77.90.31.10:4517 , piece_id=WJ7TMWHMURKZ6HJON2NBIZVIC4XEXUUR5M7KD5PFWV4MEB3DUV7A, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.520644038 (13h32m22.035746619s) [noise=true , address=104.36.82.12:4149 , piece_id=UEFYG73QPD3OQX5IF2BGE7SQHYL5OKV37LQOKGNECBGRYQLULCJQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.521219880 (13h32m22.036322466s) [noise=true , address=77.90.31.17:4771 , piece_id=YCZ26U3QI4NQ7CKNVZEEB4O6XK4ZR3SPXTKT75P5U4ZQLUWOJWQA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.609483246 (13h32m22.124585826s) [upload=13931]: Next returned nil segment Aug 7 23:11:47.609501768 (13h32m22.124604348s) [upload=13931]: Waiting for error group managing segments... Aug 7 23:11:47.614240475 (13h32m22.129343046s) [noise=true , address=45.91.116.8:4047 , piece_id=5E2VVSY65OAD6OFCQBBOTYLCLUKNVCVD3KY7OOVBI5TCYKQUKBQQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.614925369 (13h32m22.130027939s) [noise=true , address=45.91.116.5:4297 , piece_id=T76WY44JQ55TJH4KJM57QWNBP7DVYNIBDVAPVN2AUTWASN4V76IA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.615059457 (13h32m22.130162026s) [noise=true , address=45.91.116.7:4010 , piece_id=XOROULN3EEUSG45LZ5V53ORP2YV7CWLRQDSQ2AXREAFB62FGNL6A, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.615458775 (13h32m22.130561338s) [noise=true , address=77.90.31.19:4842 , piece_id=Q7PUVWG4BXEJSS4YOBVTMUHCAWWKKQAR43NH4U3R24YMK3MP6H6Q, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.616116276 (13h32m22.131218838s) [noise=true , address=77.90.31.18:4806 , piece_id=LI5HUM3HCYFYYSO6BENU4DQ2SFOPEJLRWZRM5PWCFXUZON4WCMAA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.616661348 (13h32m22.131763909s) [noise=true , address=77.90.31.17:4771 , piece_id=YCZ26U3QI4NQ7CKNVZEEB4O6XK4ZR3SPXTKT75P5U4ZQLUWOJWQA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.618000416 (13h32m22.133102978s) [noise=true , address=45.91.116.10:4081 , piece_id=WDJ6KCU4LP75N7TSSR2VTWWEON3VRWE7BFJL57H2RM7ZYI2GYLWQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.618530300 (13h32m22.133632866s) [noise=true , address=45.91.116.6:4973 , piece_id=HEA5GQ53KZVIMBZ5LC3S4Y4372WMGUSATCBDTY5BIHZ4WL2PG7HQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.619098772 (13h32m22.13420134s) [noise=true , address=77.90.31.19:4841 , piece_id=Z76CMQDB3DDLHQV35DZORZWRBX2D52BLBDCKPFXPONE3OUNGSGHA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.619697172 (13h32m22.13479975s) [noise=true , address=77.90.31.11:4553 , piece_id=NK6AJDEDIAA5N3DNKBIPKZXURRSBOJG26FIMUB2SVFTIWOKFJWNQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.620954558 (13h32m22.136057128s) [noise=true , address=77.90.31.21:4877 , piece_id=NVD7UBISRZJFOBJR7JKOX6V354Z6VCEP5TWJPGW3WOSI6S6K5FDA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.627010966 (13h32m22.142113543s) [noise=true , address=104.36.82.7:4910 , piece_id=Y75KUQR2IYRZBANBQPDFZTUJZPM54BA3KI6ZEF77NI3V755HPBOA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.629783113 (13h32m22.144885677s) [noise=true , address=77.90.31.10:4517 , piece_id=WJ7TMWHMURKZ6HJON2NBIZVIC4XEXUUR5M7KD5PFWV4MEB3DUV7A, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.629816031 (13h32m22.144918609s) [seg_pos={0 0}, upload=13931]: Segment reached optimal threshold of 13 pieces. Aug 7 23:11:47.630126623 (13h32m22.145229201s) [noise=true , address=104.36.82.14:4222 , piece_id=B5WB353FQ47THJP7MXV4AY5Z6FFSG7A66X72BLJUDK2ZTM3FNDVA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection; write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection Aug 7 23:11:47.630137290 (13h32m22.145239874s) [noise=true , address=104.36.82.12:4149 , piece_id=UEFYG73QPD3OQX5IF2BGE7SQHYL5OKV37LQOKGNECBGRYQLULCJQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection; write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection Aug 7 23:11:47.630232787 (13h32m22.145335361s) [noise=true , address=104.36.82.6:4873 , piece_id=3YGEWHQ7MXU6IPUJCYNM7L4DMNOECDSTVXMUBCX3CJCGPJ4OGGIQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: read tcp 159.223.143.39:36062->104.36.82.6:4873: use of closed network connection; context canceled Aug 7 23:11:47.630264689 (13h32m22.145367273s) [seg_pos={0 0}, upload=13931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:11:47.630345104 (13h32m22.145447682s) [upload=13931]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:11:47.630350066 (13h32m22.145452652s) [upload=13931]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:37.334152859 (13h39m11.849255428s) [upload=13931]: Done uploading segments... Aug 7 23:18:37.334173589 (13h39m11.849276166s) [upload=13931]: Upload finished. err: metaclient: context deadline exceeded Duration: 593.808282ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................XXX..X.X.X.......................] duration: 642.44689ms FAIL (2025-08-07 23:39:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 642.477657ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 233.864317ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:39:40 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 233.889272ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [........................XX.XXX..............................] duration: 198.940316ms FAIL (2025-08-07 23:39:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 30m31.135428317s) [upload=13902]: Starting upload Aug 7 23:09:56.620515140 (13h30m31.135617722s) [upload=13902]: Done starting upload Aug 7 23:09:56.620600541 (13h30m31.135703106s) [upload=13902]: Uploading segments... Aug 7 23:09:56.620663979 (13h30m31.135766559s) [upload=13902]: Got next segment. Inline: true Aug 7 23:09:56.620669313 (13h30m31.13577189s) [upload=13902]: Waiting for error group managing segments... Aug 7 23:09:56.620725144 (13h30m31.13582772s) [upload=13902]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:09:56.620729700 (13h30m31.135832276s) [upload=13902]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:09:56.620732858 (13h30m31.135835435s) [upload=13902]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:56.756931332 (13h30m31.272033912s) [upload=13902]: Done uploading segments... Aug 7 23:09:56.756952203 (13h30m31.272054766s) [upload=13902]: Upload finished. err: <nil> Aug 7 23:10:22.255312070 (13h30m56.770414636s) [upload=13913]: Starting upload Aug 7 23:10:22.255389033 (13h30m56.770491609s) [upload=13913]: Done starting upload Aug 7 23:10:22.255538785 (13h30m56.77064136s) [upload=13913]: Uploading segments... Aug 7 23:10:22.255632108 (13h30m56.77073469s) [upload=13913]: Got next segment. Inline: true Aug 7 23:10:22.255639107 (13h30m56.770741683s) [upload=13913]: Waiting for error group managing segments... Aug 7 23:10:22.255686584 (13h30m56.77078915s) [upload=13913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:22.255691754 (13h30m56.770794331s) [upload=13913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:22.255695470 (13h30m56.770798046s) [upload=13913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:22.322045125 (13h30m56.837147702s) [upload=13913]: Done uploading segments... Aug 7 23:10:22.322068753 (13h30m56.837171329s) [upload=13913]: Upload finished. err: <nil> Aug 7 23:11:00.342239204 (13h31m34.85734177s) [upload=13922]: Starting upload Aug 7 23:11:00.342310667 (13h31m34.857413235s) [upload=13922]: Done starting upload Aug 7 23:11:00.342383830 (13h31m34.85748641s) [upload=13922]: Uploading segments... Aug 7 23:11:00.342449267 (13h31m34.857551854s) [upload=13922]: Got next segment. Inline: true Aug 7 23:11:00.342455504 (13h31m34.857558092s) [upload=13922]: Waiting for error group managing segments... Aug 7 23:11:00.342503627 (13h31m34.857606209s) [upload=13922]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:00.342508465 (13h31m34.857611046s) [upload=13922]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:00.342511859 (13h31m34.857614451s) [upload=13922]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:00.401913127 (13h31m34.917015712s) [upload=13922]: Done uploading segments... Aug 7 23:11:00.401931904 (13h31m34.917034472s) [upload=13922]: Upload finished. err: <nil> Aug 7 23:11:35.523175401 (13h32m10.03827798s) [upload=13934]: Starting upload Aug 7 23:11:35.523266566 (13h32m10.038369147s) [upload=13934]: Done starting upload Aug 7 23:11:35.523355068 (13h32m10.038457647s) [upload=13934]: Uploading segments... Aug 7 23:11:35.523405664 (13h32m10.03850825s) [upload=13934]: Got next segment. Inline: true Aug 7 23:11:35.523412393 (13h32m10.038514977s) [upload=13934]: Waiting for error group managing segments... Aug 7 23:11:35.523470368 (13h32m10.038572947s) [upload=13934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:35.523479499 (13h32m10.038582083s) [upload=13934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:35.523484240 (13h32m10.038586821s) [upload=13934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:35.587656651 (13h32m10.102759216s) [upload=13934]: Done uploading segments... Aug 7 23:11:35.587674771 (13h32m10.102777338s) [upload=13934]: Upload finished. err: <nil> Aug 7 23:12:23.226340957 (13h32m57.741443533s) [upload=13947]: Starting upload Aug 7 23:12:23.226411041 (13h32m57.741513618s) [upload=13947]: Done starting upload Aug 7 23:12:23.226470575 (13h32m57.74157314s) [upload=13947]: Uploading segments... Aug 7 23:12:23.226506745 (13h32m57.741609326s) [upload=13947]: Got next segment. Inline: true Aug 7 23:12:23.226511437 (13h32m57.741614019s) [upload=13947]: Waiting for error group managing segments... Aug 7 23:12:23.226561258 (13h32m57.741663839s) [upload=13947]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:23.226566182 (13h32m57.741668762s) [upload=13947]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:23.226569380 (13h32m57.741671956s) [upload=13947]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:53.198277272 (13h33m27.713379848s) [upload=13947]: Done uploading segments... Aug 7 23:12:53.198295252 (13h33m27.713397814s) [upload=13947]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:53.310207725 (13h33m27.825310302s) [upload=13957]: Starting upload Aug 7 23:12:53.310290257 (13h33m27.82539282s) [upload=13957]: Done starting upload Aug 7 23:12:53.310379699 (13h33m27.825482276s) [upload=13957]: Uploading segments... Aug 7 23:12:53.310488747 (13h33m27.825591336s) [upload=13957]: Got next segment. Inline: true Aug 7 23:12:53.310497967 (13h33m27.825600548s) [upload=13957]: Waiting for error group managing segments... Aug 7 23:12:53.310564975 (13h33m27.825667562s) [upload=13957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:53.310572430 (13h33m27.825675017s) [upload=13957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:53.310576051 (13h33m27.825678628s) [upload=13957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:23.280527765 (13h33m57.79563034s) [upload=13957]: Done uploading segments... Aug 7 23:13:23.280561185 (13h33m57.795663762s) [upload=13957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:50.810735096 (13h34m25.325837673s) [upload=13972]: Starting upload Aug 7 23:13:50.810789852 (13h34m25.325892437s) [upload=13972]: Done starting upload Aug 7 23:13:50.810850987 (13h34m25.325953556s) [upload=13972]: Uploading segments... Aug 7 23:13:50.810923216 (13h34m25.326025797s) [upload=13972]: Got next segment. Inline: true Aug 7 23:13:50.810928523 (13h34m25.326031096s) [upload=13972]: Waiting for error group managing segments... Aug 7 23:13:50.810970440 (13h34m25.326073016s) [upload=13972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:50.810974710 (13h34m25.326077286s) [upload=13972]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:50.810978274 (13h34m25.32608085s) [upload=13972]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:50.866577223 (13h34m25.3816798s) [upload=13972]: Done uploading segments... Aug 7 23:13:50.866594654 (13h34m25.38169723s) [upload=13972]: Upload finished. err: <nil> Aug 7 23:14:54.225692687 (13h35m28.740795254s) [upload=13986]: Starting upload Aug 7 23:14:54.225785134 (13h35m28.740887721s) [upload=13986]: Done starting upload Aug 7 23:14:54.225870963 (13h35m28.740973529s) [upload=13986]: Uploading segments... Aug 7 23:14:54.225923809 (13h35m28.741026386s) [upload=13986]: Got next segment. Inline: true Aug 7 23:14:54.225928631 (13h35m28.741031207s) [upload=13986]: Waiting for error group managing segments... Aug 7 23:14:54.225969273 (13h35m28.741071842s) [upload=13986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:54.225973734 (13h35m28.7410763s) [upload=13986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:54.225976991 (13h35m28.741079566s) [upload=13986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:24.191802298 (13h35m58.706904867s) [upload=13986]: Done uploading segments... Aug 7 23:15:24.191822370 (13h35m58.70692494s) [upload=13986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:24.446619359 (13h35m58.961721924s) [upload=13992]: Starting upload Aug 7 23:15:24.446725754 (13h35m58.961828338s) [upload=13992]: Done starting upload Aug 7 23:15:24.446812887 (13h35m58.961915463s) [upload=13992]: Uploading segments... Aug 7 23:15:24.446871932 (13h35m58.961974519s) [upload=13992]: Got next segment. Inline: true Aug 7 23:15:24.446878746 (13h35m58.961981315s) [upload=13992]: Waiting for error group managing segments... Aug 7 23:15:24.446939653 (13h35m58.962042222s) [upload=13992]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:24.446945288 (13h35m58.962047863s) [upload=13992]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:24.446950467 (13h35m58.96205303s) [upload=13992]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:54.414392139 (13h36m28.929494705s) [upload=13992]: Done uploading segments... Aug 7 23:15:54.414412085 (13h36m28.929514651s) [upload=13992]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:32.150263694 (13h37m6.665366261s) [upload=14009]: Starting upload Aug 7 23:16:32.150345034 (13h37m6.665447612s) [upload=14009]: Done starting upload Aug 7 23:16:32.150402507 (13h37m6.665505071s) [upload=14009]: Uploading segments... Aug 7 23:16:32.150453597 (13h37m6.665556176s) [upload=14009]: Got next segment. Inline: true Aug 7 23:16:32.150458385 (13h37m6.665560958s) [upload=14009]: Waiting for error group managing segments... Aug 7 23:16:32.150501065 (13h37m6.665603641s) [upload=14009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:32.150505833 (13h37m6.66560841s) [upload=14009]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:32.150509597 (13h37m6.665612174s) [upload=14009]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:02.044102103 (13h37m36.55920467s) [upload=14009]: Done uploading segments... Aug 7 23:17:02.044123177 (13h37m36.559225743s) [upload=14009]: Upload finished. err: metaclient: context deadline exceeded Duration: 198.972062ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [........................XX.XXXX...X.X.......................] duration: 663.377336ms FAIL (2025-08-07 23:39:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ce_id=GH7IOGMAIMB7DEPPOCFSWD2JIW2T3GGESTVFX4YHDDODF2UKVV4A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.270963124 (13h41m24.786065727s) [noise=true , address=104.36.82.61:4398 , piece_id=6JGM2OSGIKGQJVSGKTYSKXWFWUPD2CZBJE5SUVHZTHIDXKYWSVFQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271303186 (13h41m24.786405766s) [noise=true , address=45.91.116.14:4198 , piece_id=PPA53436A4JFKNVU3PBE7WSTEWHVNITSNMU2VYCZU5DDZCFJOPSQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271556719 (13h41m24.786659309s) [noise=true , address=104.36.82.9:4034 , piece_id=UDZJLVKKAFC3IJGML35BAWKAYPYKCAGODNJBAK4R6QKEQDS5EZSA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271787526 (13h41m24.786890105s) [noise=true , address=104.36.82.42:4808 , piece_id=G6LVWECALGP45ILRGEK3X3UHOOQHP2UREVWFCDPYJZC3DFMPNYMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.272659277 (13h41m24.787761864s) [noise=true , address=64.49.13.27:4967 , piece_id=53RPNRVKZ6TYT7VP2DCXAX33YMUMDAWZS3RMRS53AW3OSEDIPCTA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.272784283 (13h41m24.787886875s) [noise=true , address=104.36.82.16:4326 , piece_id=DQUUJR7LJFQJQUKRTWXHS5SCMU4ZZ72EIYZZJY25HRM2BJYHLR7A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.273228084 (13h41m24.788330665s) [noise=true , address=104.36.82.60:4353 , piece_id=T26XKES2P2UK4QHLVWKJUZNS2IXZUYWC7FU45LGU7SC2SJILXFMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.273443857 (13h41m24.788546457s) [noise=true , address=104.36.82.44:4900 , piece_id=PS3WKIHAI4T25TOTOGXJH5RMXCCKYJQJ5JAX52ODW2G24VXLZUSQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.274091759 (13h41m24.789194346s) [noise=true , address=104.36.82.2:4759 , piece_id=WQ42DKSHG5FXUCDKII7BNEFDA25UX6Q3ARG3DXTHV7OKIBM5NHEQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.275262302 (13h41m24.790364888s) [noise=true , address=79.127.220.97:4396 , piece_id=UQGXEYE4PKBTNQO2W73CX3W5IKMVDVDDSN3ZXYOWIJGTREKQF5VQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.275755725 (13h41m24.790858314s) [noise=true , address=104.36.82.56:4236 , piece_id=PTBEE4LERZ3JO3OPPJFAIBQ5LYSTEDZCVU2FSA7QZFZJUK7Q5TQA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.276176266 (13h41m24.791278854s) [noise=true , address=104.36.82.4:4811 , piece_id=T3ZIA6J7HELFDWFFQUTJK7LGIMXOKOV5ZGURFEIKGJRBU4YWW3TA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.276690914 (13h41m24.791793502s) [noise=true , address=45.91.116.10:4104 , piece_id=ZIUMAHVVTT6X7ZVWOJI2ELFYQG7QXFP2W4CTX2PWDSCSBIZU2TMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.277946066 (13h41m24.793048658s) [noise=true , address=156.146.43.225:4447 , piece_id=QMDN5UIK3XYXB2X35EVI33U2ANOEWIU5XUU2A7CFYRSRRJLDFW4Q, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.278411190 (13h41m24.793513776s) [noise=true , address=104.36.82.23:4463 , piece_id=AF7VUSU4PDWMINEEJ3CLAQ5GYANWSBTB7ZX3KK7BBPWKX575DY5A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.363417372 (13h41m24.878519955s) [upload=14080]: Next returned nil segment Aug 7 23:20:50.363434233 (13h41m24.878536817s) [upload=14080]: Waiting for error group managing segments... Aug 7 23:20:50.370437337 (13h41m24.885539907s) [noise=true , address=45.91.116.14:4198 , piece_id=PPA53436A4JFKNVU3PBE7WSTEWHVNITSNMU2VYCZU5DDZCFJOPSQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.371668429 (13h41m24.886771s) [noise=true , address=104.36.82.56:4236 , piece_id=PTBEE4LERZ3JO3OPPJFAIBQ5LYSTEDZCVU2FSA7QZFZJUK7Q5TQA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375301858 (13h41m24.890404435s) [noise=true , address=104.36.82.23:4463 , piece_id=AF7VUSU4PDWMINEEJ3CLAQ5GYANWSBTB7ZX3KK7BBPWKX575DY5A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375604014 (13h41m24.890706595s) [noise=true , address=104.36.82.42:4808 , piece_id=G6LVWECALGP45ILRGEK3X3UHOOQHP2UREVWFCDPYJZC3DFMPNYMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375817725 (13h41m24.89092029s) [noise=true , address=104.36.82.47:4535 , piece_id=XUCXMLKMK2JHA5RRZPMYTGC5Z6JB4BFI3J6BT5RM65O4KEEI3LEA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.376297840 (13h41m24.89140042s) [noise=true , address=104.36.82.4:4811 , piece_id=T3ZIA6J7HELFDWFFQUTJK7LGIMXOKOV5ZGURFEIKGJRBU4YWW3TA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.376376567 (13h41m24.89147913s) [noise=true , address=104.36.82.9:4034 , piece_id=UDZJLVKKAFC3IJGML35BAWKAYPYKCAGODNJBAK4R6QKEQDS5EZSA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.378451878 (13h41m24.893554445s) [noise=true , address=104.36.82.37:4673 , piece_id=GH7IOGMAIMB7DEPPOCFSWD2JIW2T3GGESTVFX4YHDDODF2UKVV4A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.387814655 (13h41m24.902917224s) [noise=true , address=104.36.82.2:4759 , piece_id=WQ42DKSHG5FXUCDKII7BNEFDA25UX6Q3ARG3DXTHV7OKIBM5NHEQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.393629027 (13h41m24.908731604s) [noise=true , address=45.91.116.10:4104 , piece_id=ZIUMAHVVTT6X7ZVWOJI2ELFYQG7QXFP2W4CTX2PWDSCSBIZU2TMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.399762700 (13h41m24.914865269s) [noise=true , address=104.36.82.60:4353 , piece_id=T26XKES2P2UK4QHLVWKJUZNS2IXZUYWC7FU45LGU7SC2SJILXFMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.461291683 (13h41m24.976394246s) [noise=true , address=79.127.220.97:4396 , piece_id=UQGXEYE4PKBTNQO2W73CX3W5IKMVDVDDSN3ZXYOWIJGTREKQF5VQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.478584042 (13h41m24.993686633s) [noise=true , address=104.36.82.61:4398 , piece_id=6JGM2OSGIKGQJVSGKTYSKXWFWUPD2CZBJE5SUVHZTHIDXKYWSVFQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.520830872 (13h41m25.035933449s) [noise=true , address=156.146.43.225:4447 , piece_id=QMDN5UIK3XYXB2X35EVI33U2ANOEWIU5XUU2A7CFYRSRRJLDFW4Q, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.520859322 (13h41m25.035961898s) [seg_pos={0 0}, upload=14080]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:50.521038126 (13h41m25.036140694s) [noise=true , address=104.36.82.44:4900 , piece_id=PS3WKIHAI4T25TOTOGXJH5RMXCCKYJQJ5JAX52ODW2G24VXLZUSQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1R8K65zsCspu7W2oAnvtpz7zEsQFkDG1e5BAp6faQYMQrkh4oQ): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:50.521126310 (13h41m25.03622888s) [noise=true , address=104.36.82.5:4842 , piece_id=AOQNSYWK7J4PGSTM5JEABQ34AFL33ZNKJRI77WO6GY5RKO7DAIHQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn): protocol: write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection; write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection Aug 7 23:20:50.521313827 (13h41m25.036416407s) [noise=true , address=104.36.82.16:4326 , piece_id=DQUUJR7LJFQJQUKRTWXHS5SCMU4ZZ72EIYZZJY25HRM2BJYHLR7A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pU8MZZKS88CPFdZoaQ4msNfkCbCewTmS2wKEdN7pG5aCWoAr6): protocol: read tcp 159.223.143.39:35084->104.36.82.16:4326: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:20:50.521375844 (13h41m25.03647841s) [noise=true , address=64.49.13.27:4967 , piece_id=53RPNRVKZ6TYT7VP2DCXAX33YMUMDAWZS3RMRS53AW3OSEDIPCTA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nVt2Ze4c5jQTyqFQDsJVaZKqge6gfjwpxUZXDCx8UVcySk4r8): protocol: read tcp 159.223.143.39:54110->64.49.13.27:4967: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:20:50.521411430 (13h41m25.036513997s) [seg_pos={0 0}, upload=14080]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:50.521520073 (13h41m25.03662264s) [upload=14080]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:50.521525409 (13h41m25.036627984s) [upload=14080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.573889476 (13h41m25.088992052s) [upload=14080]: Done uploading segments... Aug 7 23:20:50.573905889 (13h41m25.089008466s) [upload=14080]: Upload finished. err: <nil> Aug 7 23:21:30.905760263 (13h42m5.42086284s) [upload=14094]: Starting upload Aug 7 23:21:30.905840100 (13h42m5.420942676s) [upload=14094]: Done starting upload Aug 7 23:21:30.905958959 (13h42m5.421061527s) [upload=14094]: Uploading segments... Aug 7 23:21:30.906533404 (13h42m5.421636042s) [upload=14094]: Got next segment. Inline: false Aug 7 23:21:30.906593461 (13h42m5.421696028s) [upload=14094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:21:30.906598444 (13h42m5.42170102s) [upload=14094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:22:30.875535565 (13h43m5.390638142s) [upload=14094]: Next segment err: context deadline exceeded Aug 7 23:22:30.875554779 (13h43m5.390657361s) [upload=14094]: Done uploading segments... Aug 7 23:22:30.875570371 (13h43m5.390672939s) [upload=14094]: Upload finished. err: context deadline exceeded Duration: 663.405769ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 28.168323ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:39:32 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 28.195834ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.......................X.XXX.XX.............................] duration: 817.462488ms FAIL (2025-08-07 23:39:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 817.483498ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........................XX..X...X..........................] duration: 256.809772ms FAIL (2025-08-07 23:39:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 162725354 (13h31m20.67782793s) [upload=13919]: Starting upload Aug 7 23:10:46.162807728 (13h31m20.677910304s) [upload=13919]: Done starting upload Aug 7 23:10:46.162877771 (13h31m20.677980347s) [upload=13919]: Uploading segments... Aug 7 23:10:46.162939319 (13h31m20.678041901s) [upload=13919]: Got next segment. Inline: true Aug 7 23:10:46.162944044 (13h31m20.678046624s) [upload=13919]: Waiting for error group managing segments... Aug 7 23:10:46.162986704 (13h31m20.678089286s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:46.162991049 (13h31m20.678093629s) [upload=13919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:46.162994213 (13h31m20.678096792s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:49.331481110 (13h31m23.84658369s) [upload=13919]: Done uploading segments... Aug 7 23:10:49.331502710 (13h31m23.846605285s) [upload=13919]: Upload finished. err: <nil> Aug 7 23:11:56.588799333 (13h32m31.103901909s) [upload=13942]: Starting upload Aug 7 23:11:56.588871172 (13h32m31.103973747s) [upload=13942]: Done starting upload Aug 7 23:11:56.588959193 (13h32m31.104061755s) [upload=13942]: Uploading segments... Aug 7 23:11:56.588996486 (13h32m31.104099072s) [upload=13942]: Got next segment. Inline: true Aug 7 23:11:56.589000914 (13h32m31.104103491s) [upload=13942]: Waiting for error group managing segments... Aug 7 23:11:56.589047870 (13h32m31.104150433s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:56.589051856 (13h32m31.104154432s) [upload=13942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:56.589054372 (13h32m31.10415695s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:56.555383755 (13h33m31.070486344s) [upload=13942]: Done uploading segments... Aug 7 23:12:56.555407795 (13h33m31.070510378s) [upload=13942]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:56.669023639 (13h33m31.184126201s) [upload=13959]: Starting upload Aug 7 23:12:56.669081512 (13h33m31.184184088s) [upload=13959]: Done starting upload Aug 7 23:12:56.669139265 (13h33m31.184241827s) [upload=13959]: Uploading segments... Aug 7 23:12:56.669181725 (13h33m31.184284307s) [upload=13959]: Got next segment. Inline: true Aug 7 23:12:56.669185727 (13h33m31.18428829s) [upload=13959]: Waiting for error group managing segments... Aug 7 23:12:56.669222629 (13h33m31.184325191s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:56.669226085 (13h33m31.18432866s) [upload=13959]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:56.669228556 (13h33m31.184331118s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.639927224 (13h34m31.155029791s) [upload=13959]: Done uploading segments... Aug 7 23:13:56.639946863 (13h34m31.155049429s) [upload=13959]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:56.754611797 (13h34m31.269714367s) [upload=13974]: Starting upload Aug 7 23:13:56.754690544 (13h34m31.269793123s) [upload=13974]: Done starting upload Aug 7 23:13:56.754787910 (13h34m31.269890478s) [upload=13974]: Uploading segments... Aug 7 23:13:56.754829279 (13h34m31.269931864s) [upload=13974]: Got next segment. Inline: true Aug 7 23:13:56.754833834 (13h34m31.269936411s) [upload=13974]: Waiting for error group managing segments... Aug 7 23:13:56.754887650 (13h34m31.269990217s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.754891993 (13h34m31.26999456s) [upload=13974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.754895495 (13h34m31.269998071s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.814228870 (13h34m31.32933145s) [upload=13974]: Done uploading segments... Aug 7 23:13:56.814253732 (13h34m31.329356295s) [upload=13974]: Upload finished. err: <nil> Aug 7 23:14:51.237083244 (13h35m25.75218581s) [upload=13984]: Starting upload Aug 7 23:14:51.237154440 (13h35m25.752257016s) [upload=13984]: Done starting upload Aug 7 23:14:51.237222602 (13h35m25.752325168s) [upload=13984]: Uploading segments... Aug 7 23:14:51.237272573 (13h35m25.752375158s) [upload=13984]: Got next segment. Inline: true Aug 7 23:14:51.237277088 (13h35m25.752379654s) [upload=13984]: Waiting for error group managing segments... Aug 7 23:14:51.237328372 (13h35m25.752430937s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:51.237332974 (13h35m25.75243555s) [upload=13984]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:51.237336117 (13h35m25.752438684s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:51.367347705 (13h35m25.882450281s) [upload=13984]: Done uploading segments... Aug 7 23:14:51.367368302 (13h35m25.882470865s) [upload=13984]: Upload finished. err: <nil> Aug 7 23:16:05.237199068 (13h36m39.752301655s) [upload=14002]: Starting upload Aug 7 23:16:05.237269888 (13h36m39.752372457s) [upload=14002]: Done starting upload Aug 7 23:16:05.237360382 (13h36m39.752462952s) [upload=14002]: Uploading segments... Aug 7 23:16:05.237841750 (13h36m39.752944336s) [upload=14002]: Got next segment. Inline: true Aug 7 23:16:05.237859632 (13h36m39.752962218s) [upload=14002]: Waiting for error group managing segments... Aug 7 23:16:05.237930004 (13h36m39.753032586s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:05.237936533 (13h36m39.753039109s) [upload=14002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:05.237940573 (13h36m39.753043148s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:05.205359206 (13h37m39.720461782s) [upload=14002]: Done uploading segments... Aug 7 23:17:05.205376470 (13h37m39.720479046s) [upload=14002]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:15.921193047 (13h37m50.436295623s) [upload=14020]: Starting upload Aug 7 23:17:15.921258731 (13h37m50.436361294s) [upload=14020]: Done starting upload Aug 7 23:17:15.921299906 (13h37m50.436402469s) [upload=14020]: Uploading segments... Aug 7 23:17:15.921331213 (13h37m50.436433799s) [upload=14020]: Got next segment. Inline: true Aug 7 23:17:15.921334685 (13h37m50.436437248s) [upload=14020]: Waiting for error group managing segments... Aug 7 23:17:15.921366597 (13h37m50.436469172s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:15.921370316 (13h37m50.436472878s) [upload=14020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:15.921372605 (13h37m50.436475167s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:15.996901858 (13h37m50.512004437s) [upload=14020]: Done uploading segments... Aug 7 23:17:15.996920869 (13h37m50.512023431s) [upload=14020]: Upload finished. err: <nil> Aug 7 23:18:10.846312036 (13h38m45.361414612s) [upload=14034]: Starting upload Aug 7 23:18:10.846377244 (13h38m45.36147981s) [upload=14034]: Done starting upload Aug 7 23:18:10.846461339 (13h38m45.361563904s) [upload=14034]: Uploading segments... Aug 7 23:18:10.846496473 (13h38m45.361599058s) [upload=14034]: Got next segment. Inline: true Aug 7 23:18:10.846501164 (13h38m45.36160374s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:18:10.846554786 (13h38m45.361657362s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:10.846559294 (13h38m45.361661871s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:10.846562922 (13h38m45.361665498s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:11.200018957 (13h38m45.715121534s) [upload=14034]: Done uploading segments... Aug 7 23:18:11.200049582 (13h38m45.715152158s) [upload=14034]: Upload finished. err: <nil> Aug 7 23:19:15.896167300 (13h39m50.411269878s) [upload=14051]: Starting upload Aug 7 23:19:15.896353995 (13h39m50.411456637s) [upload=14051]: Done starting upload Aug 7 23:19:15.896436268 (13h39m50.411538832s) [upload=14051]: Uploading segments... Aug 7 23:19:15.896501153 (13h39m50.411603732s) [upload=14051]: Got next segment. Inline: true Aug 7 23:19:15.896505978 (13h39m50.411608555s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:19:15.896570054 (13h39m50.411672631s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:15.896574888 (13h39m50.411677465s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:15.896578764 (13h39m50.411681341s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:15.976040109 (13h39m50.491142676s) [upload=14051]: Done uploading segments... Aug 7 23:19:15.976057062 (13h39m50.491159637s) [upload=14051]: Upload finished. err: <nil> Aug 7 23:19:50.253965481 (13h40m24.76906806s) [upload=14062]: Starting upload Aug 7 23:19:50.254050632 (13h40m24.769153219s) [upload=14062]: Done starting upload Aug 7 23:19:50.254136562 (13h40m24.769239133s) [upload=14062]: Uploading segments... Aug 7 23:19:50.254189310 (13h40m24.769291896s) [upload=14062]: Got next segment. Inline: true Aug 7 23:19:50.254194507 (13h40m24.769297084s) [upload=14062]: Waiting for error group managing segments... Aug 7 23:19:50.254255974 (13h40m24.769358555s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:50.254262645 (13h40m24.769365222s) [upload=14062]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:50.254267031 (13h40m24.769369607s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.224097554 (13h41m24.739200131s) [upload=14062]: Done uploading segments... Aug 7 23:20:50.224116508 (13h41m24.739219086s) [upload=14062]: Upload finished. err: metaclient: context deadline exceeded Duration: 256.836432ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [........................X..XXXXX.X.XX.......................] duration: 235.490244ms FAIL (2025-08-07 23:39:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 235.519977ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [........................XX.X.X....XX........................] duration: 273.046872ms FAIL (2025-08-07 23:39:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 273.608265ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.........................XX.XXX.............................] duration: 230.014018ms FAIL (2025-08-07 23:39:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 30m31.135428317s) [upload=13902]: Starting upload Aug 7 23:09:56.620515140 (13h30m31.135617722s) [upload=13902]: Done starting upload Aug 7 23:09:56.620600541 (13h30m31.135703106s) [upload=13902]: Uploading segments... Aug 7 23:09:56.620663979 (13h30m31.135766559s) [upload=13902]: Got next segment. Inline: true Aug 7 23:09:56.620669313 (13h30m31.13577189s) [upload=13902]: Waiting for error group managing segments... Aug 7 23:09:56.620725144 (13h30m31.13582772s) [upload=13902]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:09:56.620729700 (13h30m31.135832276s) [upload=13902]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:09:56.620732858 (13h30m31.135835435s) [upload=13902]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:56.756931332 (13h30m31.272033912s) [upload=13902]: Done uploading segments... Aug 7 23:09:56.756952203 (13h30m31.272054766s) [upload=13902]: Upload finished. err: <nil> Aug 7 23:10:22.255312070 (13h30m56.770414636s) [upload=13913]: Starting upload Aug 7 23:10:22.255389033 (13h30m56.770491609s) [upload=13913]: Done starting upload Aug 7 23:10:22.255538785 (13h30m56.77064136s) [upload=13913]: Uploading segments... Aug 7 23:10:22.255632108 (13h30m56.77073469s) [upload=13913]: Got next segment. Inline: true Aug 7 23:10:22.255639107 (13h30m56.770741683s) [upload=13913]: Waiting for error group managing segments... Aug 7 23:10:22.255686584 (13h30m56.77078915s) [upload=13913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:22.255691754 (13h30m56.770794331s) [upload=13913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:22.255695470 (13h30m56.770798046s) [upload=13913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:22.322045125 (13h30m56.837147702s) [upload=13913]: Done uploading segments... Aug 7 23:10:22.322068753 (13h30m56.837171329s) [upload=13913]: Upload finished. err: <nil> Aug 7 23:11:00.342239204 (13h31m34.85734177s) [upload=13922]: Starting upload Aug 7 23:11:00.342310667 (13h31m34.857413235s) [upload=13922]: Done starting upload Aug 7 23:11:00.342383830 (13h31m34.85748641s) [upload=13922]: Uploading segments... Aug 7 23:11:00.342449267 (13h31m34.857551854s) [upload=13922]: Got next segment. Inline: true Aug 7 23:11:00.342455504 (13h31m34.857558092s) [upload=13922]: Waiting for error group managing segments... Aug 7 23:11:00.342503627 (13h31m34.857606209s) [upload=13922]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:00.342508465 (13h31m34.857611046s) [upload=13922]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:00.342511859 (13h31m34.857614451s) [upload=13922]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:00.401913127 (13h31m34.917015712s) [upload=13922]: Done uploading segments... Aug 7 23:11:00.401931904 (13h31m34.917034472s) [upload=13922]: Upload finished. err: <nil> Aug 7 23:11:35.523175401 (13h32m10.03827798s) [upload=13934]: Starting upload Aug 7 23:11:35.523266566 (13h32m10.038369147s) [upload=13934]: Done starting upload Aug 7 23:11:35.523355068 (13h32m10.038457647s) [upload=13934]: Uploading segments... Aug 7 23:11:35.523405664 (13h32m10.03850825s) [upload=13934]: Got next segment. Inline: true Aug 7 23:11:35.523412393 (13h32m10.038514977s) [upload=13934]: Waiting for error group managing segments... Aug 7 23:11:35.523470368 (13h32m10.038572947s) [upload=13934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:35.523479499 (13h32m10.038582083s) [upload=13934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:35.523484240 (13h32m10.038586821s) [upload=13934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:35.587656651 (13h32m10.102759216s) [upload=13934]: Done uploading segments... Aug 7 23:11:35.587674771 (13h32m10.102777338s) [upload=13934]: Upload finished. err: <nil> Aug 7 23:12:23.226340957 (13h32m57.741443533s) [upload=13947]: Starting upload Aug 7 23:12:23.226411041 (13h32m57.741513618s) [upload=13947]: Done starting upload Aug 7 23:12:23.226470575 (13h32m57.74157314s) [upload=13947]: Uploading segments... Aug 7 23:12:23.226506745 (13h32m57.741609326s) [upload=13947]: Got next segment. Inline: true Aug 7 23:12:23.226511437 (13h32m57.741614019s) [upload=13947]: Waiting for error group managing segments... Aug 7 23:12:23.226561258 (13h32m57.741663839s) [upload=13947]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:23.226566182 (13h32m57.741668762s) [upload=13947]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:23.226569380 (13h32m57.741671956s) [upload=13947]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:53.198277272 (13h33m27.713379848s) [upload=13947]: Done uploading segments... Aug 7 23:12:53.198295252 (13h33m27.713397814s) [upload=13947]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:53.310207725 (13h33m27.825310302s) [upload=13957]: Starting upload Aug 7 23:12:53.310290257 (13h33m27.82539282s) [upload=13957]: Done starting upload Aug 7 23:12:53.310379699 (13h33m27.825482276s) [upload=13957]: Uploading segments... Aug 7 23:12:53.310488747 (13h33m27.825591336s) [upload=13957]: Got next segment. Inline: true Aug 7 23:12:53.310497967 (13h33m27.825600548s) [upload=13957]: Waiting for error group managing segments... Aug 7 23:12:53.310564975 (13h33m27.825667562s) [upload=13957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:53.310572430 (13h33m27.825675017s) [upload=13957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:53.310576051 (13h33m27.825678628s) [upload=13957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:23.280527765 (13h33m57.79563034s) [upload=13957]: Done uploading segments... Aug 7 23:13:23.280561185 (13h33m57.795663762s) [upload=13957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:50.810735096 (13h34m25.325837673s) [upload=13972]: Starting upload Aug 7 23:13:50.810789852 (13h34m25.325892437s) [upload=13972]: Done starting upload Aug 7 23:13:50.810850987 (13h34m25.325953556s) [upload=13972]: Uploading segments... Aug 7 23:13:50.810923216 (13h34m25.326025797s) [upload=13972]: Got next segment. Inline: true Aug 7 23:13:50.810928523 (13h34m25.326031096s) [upload=13972]: Waiting for error group managing segments... Aug 7 23:13:50.810970440 (13h34m25.326073016s) [upload=13972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:50.810974710 (13h34m25.326077286s) [upload=13972]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:50.810978274 (13h34m25.32608085s) [upload=13972]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:50.866577223 (13h34m25.3816798s) [upload=13972]: Done uploading segments... Aug 7 23:13:50.866594654 (13h34m25.38169723s) [upload=13972]: Upload finished. err: <nil> Aug 7 23:14:54.225692687 (13h35m28.740795254s) [upload=13986]: Starting upload Aug 7 23:14:54.225785134 (13h35m28.740887721s) [upload=13986]: Done starting upload Aug 7 23:14:54.225870963 (13h35m28.740973529s) [upload=13986]: Uploading segments... Aug 7 23:14:54.225923809 (13h35m28.741026386s) [upload=13986]: Got next segment. Inline: true Aug 7 23:14:54.225928631 (13h35m28.741031207s) [upload=13986]: Waiting for error group managing segments... Aug 7 23:14:54.225969273 (13h35m28.741071842s) [upload=13986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:54.225973734 (13h35m28.7410763s) [upload=13986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:54.225976991 (13h35m28.741079566s) [upload=13986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:24.191802298 (13h35m58.706904867s) [upload=13986]: Done uploading segments... Aug 7 23:15:24.191822370 (13h35m58.70692494s) [upload=13986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:24.446619359 (13h35m58.961721924s) [upload=13992]: Starting upload Aug 7 23:15:24.446725754 (13h35m58.961828338s) [upload=13992]: Done starting upload Aug 7 23:15:24.446812887 (13h35m58.961915463s) [upload=13992]: Uploading segments... Aug 7 23:15:24.446871932 (13h35m58.961974519s) [upload=13992]: Got next segment. Inline: true Aug 7 23:15:24.446878746 (13h35m58.961981315s) [upload=13992]: Waiting for error group managing segments... Aug 7 23:15:24.446939653 (13h35m58.962042222s) [upload=13992]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:24.446945288 (13h35m58.962047863s) [upload=13992]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:24.446950467 (13h35m58.96205303s) [upload=13992]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:54.414392139 (13h36m28.929494705s) [upload=13992]: Done uploading segments... Aug 7 23:15:54.414412085 (13h36m28.929514651s) [upload=13992]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:32.150263694 (13h37m6.665366261s) [upload=14009]: Starting upload Aug 7 23:16:32.150345034 (13h37m6.665447612s) [upload=14009]: Done starting upload Aug 7 23:16:32.150402507 (13h37m6.665505071s) [upload=14009]: Uploading segments... Aug 7 23:16:32.150453597 (13h37m6.665556176s) [upload=14009]: Got next segment. Inline: true Aug 7 23:16:32.150458385 (13h37m6.665560958s) [upload=14009]: Waiting for error group managing segments... Aug 7 23:16:32.150501065 (13h37m6.665603641s) [upload=14009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:32.150505833 (13h37m6.66560841s) [upload=14009]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:32.150509597 (13h37m6.665612174s) [upload=14009]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:02.044102103 (13h37m36.55920467s) [upload=14009]: Done uploading segments... Aug 7 23:17:02.044123177 (13h37m36.559225743s) [upload=14009]: Upload finished. err: metaclient: context deadline exceeded Duration: 230.043874ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................XXXXX...............................] duration: 1.795037339s FAIL (2025-08-07 23:39:16 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.795065482s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [........................X.XXX.XX............................] duration: 773.022319ms FAIL (2025-08-07 23:38:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 773.072928ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 39.441327ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:38:56 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 39.46254ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.........................XX.X.X....XX.......................] duration: 287.309621ms FAIL (2025-08-07 23:38:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 287.345204ms edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 31.357681ms target: https://us1.storj.io/login FAIL (2025-08-07 23:38:50 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 31.38514ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [.........................X..XXXXX.X.XX......................] duration: 219.668897ms FAIL (2025-08-07 23:38:46 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 219.770385ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 498.281567ms FAIL (2025-08-07 23:38:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:11:04.814410393 (13h31m39.329512969s) [upload=13925]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:11:04.814415016 (13h31m39.329517592s) [upload=13925]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:04.947583472 (13h31m39.462686071s) [upload=13925]: Done uploading segments... Aug 7 23:11:04.947609736 (13h31m39.462712324s) [upload=13925]: Upload finished. err: <nil> Aug 7 23:11:29.227536921 (13h32m3.742639487s) [upload=13931]: Starting upload Aug 7 23:11:29.227600047 (13h32m3.742702623s) [upload=13931]: Done starting upload Aug 7 23:11:29.227715021 (13h32m3.742817591s) [upload=13931]: Uploading segments... Aug 7 23:11:29.227850209 (13h32m3.74295278s) [upload=13931]: Got next segment. Inline: false Aug 7 23:11:29.227898592 (13h32m3.743001169s) [upload=13931]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:29.227903365 (13h32m3.743005942s) [upload=13931]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:11:47.514990540 (13h32m22.03009312s) [seg_pos={0 0}, upload=13931]: Begin upload segment... Aug 7 23:11:47.515067094 (13h32m22.030169657s) [seg_pos={0 0}, upload=13931]: Done begin upload segment. Aug 7 23:11:47.515129228 (13h32m22.030231814s) [noise=true , address=77.90.31.18:4806 , piece_id=LI5HUM3HCYFYYSO6BENU4DQ2SFOPEJLRWZRM5PWCFXUZON4WCMAA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.515554241 (13h32m22.03065682s) [noise=true , address=104.36.82.14:4222 , piece_id=B5WB353FQ47THJP7MXV4AY5Z6FFSG7A66X72BLJUDK2ZTM3FNDVA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.516068554 (13h32m22.031171143s) [noise=true , address=45.91.116.7:4010 , piece_id=XOROULN3EEUSG45LZ5V53ORP2YV7CWLRQDSQ2AXREAFB62FGNL6A, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.516625763 (13h32m22.031728346s) [noise=true , address=45.91.116.8:4047 , piece_id=5E2VVSY65OAD6OFCQBBOTYLCLUKNVCVD3KY7OOVBI5TCYKQUKBQQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517060179 (13h32m22.03216276s) [noise=true , address=104.36.82.7:4910 , piece_id=Y75KUQR2IYRZBANBQPDFZTUJZPM54BA3KI6ZEF77NI3V755HPBOA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517396640 (13h32m22.032499217s) [noise=true , address=104.36.82.6:4873 , piece_id=3YGEWHQ7MXU6IPUJCYNM7L4DMNOECDSTVXMUBCX3CJCGPJ4OGGIQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.517801490 (13h32m22.032904072s) [noise=true , address=45.91.116.10:4081 , piece_id=WDJ6KCU4LP75N7TSSR2VTWWEON3VRWE7BFJL57H2RM7ZYI2GYLWQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518135215 (13h32m22.033237797s) [noise=true , address=45.91.116.5:4297 , piece_id=T76WY44JQ55TJH4KJM57QWNBP7DVYNIBDVAPVN2AUTWASN4V76IA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518441967 (13h32m22.033544543s) [noise=true , address=77.90.31.11:4553 , piece_id=NK6AJDEDIAA5N3DNKBIPKZXURRSBOJG26FIMUB2SVFTIWOKFJWNQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.518756911 (13h32m22.033859486s) [noise=true , address=77.90.31.21:4877 , piece_id=NVD7UBISRZJFOBJR7JKOX6V354Z6VCEP5TWJPGW3WOSI6S6K5FDA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519217404 (13h32m22.034319986s) [noise=true , address=45.91.116.6:4973 , piece_id=HEA5GQ53KZVIMBZ5LC3S4Y4372WMGUSATCBDTY5BIHZ4WL2PG7HQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519516685 (13h32m22.034619271s) [noise=true , address=77.90.31.19:4841 , piece_id=Z76CMQDB3DDLHQV35DZORZWRBX2D52BLBDCKPFXPONE3OUNGSGHA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.519904148 (13h32m22.035006727s) [noise=true , address=77.90.31.19:4842 , piece_id=Q7PUVWG4BXEJSS4YOBVTMUHCAWWKKQAR43NH4U3R24YMK3MP6H6Q, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.520220651 (13h32m22.035323239s) [noise=true , address=77.90.31.10:4517 , piece_id=WJ7TMWHMURKZ6HJON2NBIZVIC4XEXUUR5M7KD5PFWV4MEB3DUV7A, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.520644038 (13h32m22.035746619s) [noise=true , address=104.36.82.12:4149 , piece_id=UEFYG73QPD3OQX5IF2BGE7SQHYL5OKV37LQOKGNECBGRYQLULCJQ, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.521219880 (13h32m22.036322466s) [noise=true , address=77.90.31.17:4771 , piece_id=YCZ26U3QI4NQ7CKNVZEEB4O6XK4ZR3SPXTKT75P5U4ZQLUWOJWQA, seg_pos={0 0}, upload=13931]: Uploading piece... Aug 7 23:11:47.609483246 (13h32m22.124585826s) [upload=13931]: Next returned nil segment Aug 7 23:11:47.609501768 (13h32m22.124604348s) [upload=13931]: Waiting for error group managing segments... Aug 7 23:11:47.614240475 (13h32m22.129343046s) [noise=true , address=45.91.116.8:4047 , piece_id=5E2VVSY65OAD6OFCQBBOTYLCLUKNVCVD3KY7OOVBI5TCYKQUKBQQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.614925369 (13h32m22.130027939s) [noise=true , address=45.91.116.5:4297 , piece_id=T76WY44JQ55TJH4KJM57QWNBP7DVYNIBDVAPVN2AUTWASN4V76IA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.615059457 (13h32m22.130162026s) [noise=true , address=45.91.116.7:4010 , piece_id=XOROULN3EEUSG45LZ5V53ORP2YV7CWLRQDSQ2AXREAFB62FGNL6A, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.615458775 (13h32m22.130561338s) [noise=true , address=77.90.31.19:4842 , piece_id=Q7PUVWG4BXEJSS4YOBVTMUHCAWWKKQAR43NH4U3R24YMK3MP6H6Q, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.616116276 (13h32m22.131218838s) [noise=true , address=77.90.31.18:4806 , piece_id=LI5HUM3HCYFYYSO6BENU4DQ2SFOPEJLRWZRM5PWCFXUZON4WCMAA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.616661348 (13h32m22.131763909s) [noise=true , address=77.90.31.17:4771 , piece_id=YCZ26U3QI4NQ7CKNVZEEB4O6XK4ZR3SPXTKT75P5U4ZQLUWOJWQA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.618000416 (13h32m22.133102978s) [noise=true , address=45.91.116.10:4081 , piece_id=WDJ6KCU4LP75N7TSSR2VTWWEON3VRWE7BFJL57H2RM7ZYI2GYLWQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.618530300 (13h32m22.133632866s) [noise=true , address=45.91.116.6:4973 , piece_id=HEA5GQ53KZVIMBZ5LC3S4Y4372WMGUSATCBDTY5BIHZ4WL2PG7HQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.619098772 (13h32m22.13420134s) [noise=true , address=77.90.31.19:4841 , piece_id=Z76CMQDB3DDLHQV35DZORZWRBX2D52BLBDCKPFXPONE3OUNGSGHA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.619697172 (13h32m22.13479975s) [noise=true , address=77.90.31.11:4553 , piece_id=NK6AJDEDIAA5N3DNKBIPKZXURRSBOJG26FIMUB2SVFTIWOKFJWNQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.620954558 (13h32m22.136057128s) [noise=true , address=77.90.31.21:4877 , piece_id=NVD7UBISRZJFOBJR7JKOX6V354Z6VCEP5TWJPGW3WOSI6S6K5FDA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.627010966 (13h32m22.142113543s) [noise=true , address=104.36.82.7:4910 , piece_id=Y75KUQR2IYRZBANBQPDFZTUJZPM54BA3KI6ZEF77NI3V755HPBOA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.629783113 (13h32m22.144885677s) [noise=true , address=77.90.31.10:4517 , piece_id=WJ7TMWHMURKZ6HJON2NBIZVIC4XEXUUR5M7KD5PFWV4MEB3DUV7A, seg_pos={0 0}, upload=13931]: Done uploading piece. err: <nil> Aug 7 23:11:47.629816031 (13h32m22.144918609s) [seg_pos={0 0}, upload=13931]: Segment reached optimal threshold of 13 pieces. Aug 7 23:11:47.630126623 (13h32m22.145229201s) [noise=true , address=104.36.82.14:4222 , piece_id=B5WB353FQ47THJP7MXV4AY5Z6FFSG7A66X72BLJUDK2ZTM3FNDVA, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection; write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47404->104.36.82.14:4222: use of closed network connection Aug 7 23:11:47.630137290 (13h32m22.145239874s) [noise=true , address=104.36.82.12:4149 , piece_id=UEFYG73QPD3OQX5IF2BGE7SQHYL5OKV37LQOKGNECBGRYQLULCJQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection; write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:35622->104.36.82.12:4149: use of closed network connection Aug 7 23:11:47.630232787 (13h32m22.145335361s) [noise=true , address=104.36.82.6:4873 , piece_id=3YGEWHQ7MXU6IPUJCYNM7L4DMNOECDSTVXMUBCX3CJCGPJ4OGGIQ, seg_pos={0 0}, upload=13931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: read tcp 159.223.143.39:36062->104.36.82.6:4873: use of closed network connection; context canceled Aug 7 23:11:47.630264689 (13h32m22.145367273s) [seg_pos={0 0}, upload=13931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:11:47.630345104 (13h32m22.145447682s) [upload=13931]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:11:47.630350066 (13h32m22.145452652s) [upload=13931]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:37.334152859 (13h39m11.849255428s) [upload=13931]: Done uploading segments... Aug 7 23:18:37.334173589 (13h39m11.849276166s) [upload=13931]: Upload finished. err: metaclient: context deadline exceeded Duration: 498.308748ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................XXXXX..............................] duration: 1.869436744s FAIL (2025-08-07 23:38:40 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.869467718s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................XXX..X.X.X......................] duration: 571.397944ms FAIL (2025-08-07 23:38:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 571.478034ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 98.977604ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:38:29 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 99.004901ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........................XX.XXXX...X.X......................] duration: 973.303184ms FAIL (2025-08-07 23:38:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ce_id=GH7IOGMAIMB7DEPPOCFSWD2JIW2T3GGESTVFX4YHDDODF2UKVV4A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.270963124 (13h41m24.786065727s) [noise=true , address=104.36.82.61:4398 , piece_id=6JGM2OSGIKGQJVSGKTYSKXWFWUPD2CZBJE5SUVHZTHIDXKYWSVFQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271303186 (13h41m24.786405766s) [noise=true , address=45.91.116.14:4198 , piece_id=PPA53436A4JFKNVU3PBE7WSTEWHVNITSNMU2VYCZU5DDZCFJOPSQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271556719 (13h41m24.786659309s) [noise=true , address=104.36.82.9:4034 , piece_id=UDZJLVKKAFC3IJGML35BAWKAYPYKCAGODNJBAK4R6QKEQDS5EZSA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.271787526 (13h41m24.786890105s) [noise=true , address=104.36.82.42:4808 , piece_id=G6LVWECALGP45ILRGEK3X3UHOOQHP2UREVWFCDPYJZC3DFMPNYMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.272659277 (13h41m24.787761864s) [noise=true , address=64.49.13.27:4967 , piece_id=53RPNRVKZ6TYT7VP2DCXAX33YMUMDAWZS3RMRS53AW3OSEDIPCTA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.272784283 (13h41m24.787886875s) [noise=true , address=104.36.82.16:4326 , piece_id=DQUUJR7LJFQJQUKRTWXHS5SCMU4ZZ72EIYZZJY25HRM2BJYHLR7A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.273228084 (13h41m24.788330665s) [noise=true , address=104.36.82.60:4353 , piece_id=T26XKES2P2UK4QHLVWKJUZNS2IXZUYWC7FU45LGU7SC2SJILXFMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.273443857 (13h41m24.788546457s) [noise=true , address=104.36.82.44:4900 , piece_id=PS3WKIHAI4T25TOTOGXJH5RMXCCKYJQJ5JAX52ODW2G24VXLZUSQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.274091759 (13h41m24.789194346s) [noise=true , address=104.36.82.2:4759 , piece_id=WQ42DKSHG5FXUCDKII7BNEFDA25UX6Q3ARG3DXTHV7OKIBM5NHEQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.275262302 (13h41m24.790364888s) [noise=true , address=79.127.220.97:4396 , piece_id=UQGXEYE4PKBTNQO2W73CX3W5IKMVDVDDSN3ZXYOWIJGTREKQF5VQ, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.275755725 (13h41m24.790858314s) [noise=true , address=104.36.82.56:4236 , piece_id=PTBEE4LERZ3JO3OPPJFAIBQ5LYSTEDZCVU2FSA7QZFZJUK7Q5TQA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.276176266 (13h41m24.791278854s) [noise=true , address=104.36.82.4:4811 , piece_id=T3ZIA6J7HELFDWFFQUTJK7LGIMXOKOV5ZGURFEIKGJRBU4YWW3TA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.276690914 (13h41m24.791793502s) [noise=true , address=45.91.116.10:4104 , piece_id=ZIUMAHVVTT6X7ZVWOJI2ELFYQG7QXFP2W4CTX2PWDSCSBIZU2TMA, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.277946066 (13h41m24.793048658s) [noise=true , address=156.146.43.225:4447 , piece_id=QMDN5UIK3XYXB2X35EVI33U2ANOEWIU5XUU2A7CFYRSRRJLDFW4Q, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.278411190 (13h41m24.793513776s) [noise=true , address=104.36.82.23:4463 , piece_id=AF7VUSU4PDWMINEEJ3CLAQ5GYANWSBTB7ZX3KK7BBPWKX575DY5A, seg_pos={0 0}, upload=14080]: Uploading piece... Aug 7 23:20:50.363417372 (13h41m24.878519955s) [upload=14080]: Next returned nil segment Aug 7 23:20:50.363434233 (13h41m24.878536817s) [upload=14080]: Waiting for error group managing segments... Aug 7 23:20:50.370437337 (13h41m24.885539907s) [noise=true , address=45.91.116.14:4198 , piece_id=PPA53436A4JFKNVU3PBE7WSTEWHVNITSNMU2VYCZU5DDZCFJOPSQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.371668429 (13h41m24.886771s) [noise=true , address=104.36.82.56:4236 , piece_id=PTBEE4LERZ3JO3OPPJFAIBQ5LYSTEDZCVU2FSA7QZFZJUK7Q5TQA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375301858 (13h41m24.890404435s) [noise=true , address=104.36.82.23:4463 , piece_id=AF7VUSU4PDWMINEEJ3CLAQ5GYANWSBTB7ZX3KK7BBPWKX575DY5A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375604014 (13h41m24.890706595s) [noise=true , address=104.36.82.42:4808 , piece_id=G6LVWECALGP45ILRGEK3X3UHOOQHP2UREVWFCDPYJZC3DFMPNYMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.375817725 (13h41m24.89092029s) [noise=true , address=104.36.82.47:4535 , piece_id=XUCXMLKMK2JHA5RRZPMYTGC5Z6JB4BFI3J6BT5RM65O4KEEI3LEA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.376297840 (13h41m24.89140042s) [noise=true , address=104.36.82.4:4811 , piece_id=T3ZIA6J7HELFDWFFQUTJK7LGIMXOKOV5ZGURFEIKGJRBU4YWW3TA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.376376567 (13h41m24.89147913s) [noise=true , address=104.36.82.9:4034 , piece_id=UDZJLVKKAFC3IJGML35BAWKAYPYKCAGODNJBAK4R6QKEQDS5EZSA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.378451878 (13h41m24.893554445s) [noise=true , address=104.36.82.37:4673 , piece_id=GH7IOGMAIMB7DEPPOCFSWD2JIW2T3GGESTVFX4YHDDODF2UKVV4A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.387814655 (13h41m24.902917224s) [noise=true , address=104.36.82.2:4759 , piece_id=WQ42DKSHG5FXUCDKII7BNEFDA25UX6Q3ARG3DXTHV7OKIBM5NHEQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.393629027 (13h41m24.908731604s) [noise=true , address=45.91.116.10:4104 , piece_id=ZIUMAHVVTT6X7ZVWOJI2ELFYQG7QXFP2W4CTX2PWDSCSBIZU2TMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.399762700 (13h41m24.914865269s) [noise=true , address=104.36.82.60:4353 , piece_id=T26XKES2P2UK4QHLVWKJUZNS2IXZUYWC7FU45LGU7SC2SJILXFMA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.461291683 (13h41m24.976394246s) [noise=true , address=79.127.220.97:4396 , piece_id=UQGXEYE4PKBTNQO2W73CX3W5IKMVDVDDSN3ZXYOWIJGTREKQF5VQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.478584042 (13h41m24.993686633s) [noise=true , address=104.36.82.61:4398 , piece_id=6JGM2OSGIKGQJVSGKTYSKXWFWUPD2CZBJE5SUVHZTHIDXKYWSVFQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.520830872 (13h41m25.035933449s) [noise=true , address=156.146.43.225:4447 , piece_id=QMDN5UIK3XYXB2X35EVI33U2ANOEWIU5XUU2A7CFYRSRRJLDFW4Q, seg_pos={0 0}, upload=14080]: Done uploading piece. err: <nil> Aug 7 23:20:50.520859322 (13h41m25.035961898s) [seg_pos={0 0}, upload=14080]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:50.521038126 (13h41m25.036140694s) [noise=true , address=104.36.82.44:4900 , piece_id=PS3WKIHAI4T25TOTOGXJH5RMXCCKYJQJ5JAX52ODW2G24VXLZUSQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1R8K65zsCspu7W2oAnvtpz7zEsQFkDG1e5BAp6faQYMQrkh4oQ): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:50.521126310 (13h41m25.03622888s) [noise=true , address=104.36.82.5:4842 , piece_id=AOQNSYWK7J4PGSTM5JEABQ34AFL33ZNKJRI77WO6GY5RKO7DAIHQ, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn): protocol: write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection; write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:48910->104.36.82.5:4842: use of closed network connection Aug 7 23:20:50.521313827 (13h41m25.036416407s) [noise=true , address=104.36.82.16:4326 , piece_id=DQUUJR7LJFQJQUKRTWXHS5SCMU4ZZ72EIYZZJY25HRM2BJYHLR7A, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pU8MZZKS88CPFdZoaQ4msNfkCbCewTmS2wKEdN7pG5aCWoAr6): protocol: read tcp 159.223.143.39:35084->104.36.82.16:4326: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:20:50.521375844 (13h41m25.03647841s) [noise=true , address=64.49.13.27:4967 , piece_id=53RPNRVKZ6TYT7VP2DCXAX33YMUMDAWZS3RMRS53AW3OSEDIPCTA, seg_pos={0 0}, upload=14080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nVt2Ze4c5jQTyqFQDsJVaZKqge6gfjwpxUZXDCx8UVcySk4r8): protocol: read tcp 159.223.143.39:54110->64.49.13.27:4967: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:20:50.521411430 (13h41m25.036513997s) [seg_pos={0 0}, upload=14080]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:50.521520073 (13h41m25.03662264s) [upload=14080]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:50.521525409 (13h41m25.036627984s) [upload=14080]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.573889476 (13h41m25.088992052s) [upload=14080]: Done uploading segments... Aug 7 23:20:50.573905889 (13h41m25.089008466s) [upload=14080]: Upload finished. err: <nil> Aug 7 23:21:30.905760263 (13h42m5.42086284s) [upload=14094]: Starting upload Aug 7 23:21:30.905840100 (13h42m5.420942676s) [upload=14094]: Done starting upload Aug 7 23:21:30.905958959 (13h42m5.421061527s) [upload=14094]: Uploading segments... Aug 7 23:21:30.906533404 (13h42m5.421636042s) [upload=14094]: Got next segment. Inline: false Aug 7 23:21:30.906593461 (13h42m5.421696028s) [upload=14094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:21:30.906598444 (13h42m5.42170102s) [upload=14094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:22:30.875535565 (13h43m5.390638142s) [upload=14094]: Next segment err: context deadline exceeded Aug 7 23:22:30.875554779 (13h43m5.390657361s) [upload=14094]: Done uploading segments... Aug 7 23:22:30.875570371 (13h43m5.390672939s) [upload=14094]: Upload finished. err: context deadline exceeded Duration: 973.333652ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 29.035473ms target: https://us1.storj.io/login FAIL (2025-08-07 23:38:23 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 29.080592ms core satellite us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 36.602193ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:38:22 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 36.631384ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........................XX..X...X.........................] duration: 268.095332ms FAIL (2025-08-07 23:38:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 162725354 (13h31m20.67782793s) [upload=13919]: Starting upload Aug 7 23:10:46.162807728 (13h31m20.677910304s) [upload=13919]: Done starting upload Aug 7 23:10:46.162877771 (13h31m20.677980347s) [upload=13919]: Uploading segments... Aug 7 23:10:46.162939319 (13h31m20.678041901s) [upload=13919]: Got next segment. Inline: true Aug 7 23:10:46.162944044 (13h31m20.678046624s) [upload=13919]: Waiting for error group managing segments... Aug 7 23:10:46.162986704 (13h31m20.678089286s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:46.162991049 (13h31m20.678093629s) [upload=13919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:46.162994213 (13h31m20.678096792s) [upload=13919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:49.331481110 (13h31m23.84658369s) [upload=13919]: Done uploading segments... Aug 7 23:10:49.331502710 (13h31m23.846605285s) [upload=13919]: Upload finished. err: <nil> Aug 7 23:11:56.588799333 (13h32m31.103901909s) [upload=13942]: Starting upload Aug 7 23:11:56.588871172 (13h32m31.103973747s) [upload=13942]: Done starting upload Aug 7 23:11:56.588959193 (13h32m31.104061755s) [upload=13942]: Uploading segments... Aug 7 23:11:56.588996486 (13h32m31.104099072s) [upload=13942]: Got next segment. Inline: true Aug 7 23:11:56.589000914 (13h32m31.104103491s) [upload=13942]: Waiting for error group managing segments... Aug 7 23:11:56.589047870 (13h32m31.104150433s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:56.589051856 (13h32m31.104154432s) [upload=13942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:56.589054372 (13h32m31.10415695s) [upload=13942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:56.555383755 (13h33m31.070486344s) [upload=13942]: Done uploading segments... Aug 7 23:12:56.555407795 (13h33m31.070510378s) [upload=13942]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:56.669023639 (13h33m31.184126201s) [upload=13959]: Starting upload Aug 7 23:12:56.669081512 (13h33m31.184184088s) [upload=13959]: Done starting upload Aug 7 23:12:56.669139265 (13h33m31.184241827s) [upload=13959]: Uploading segments... Aug 7 23:12:56.669181725 (13h33m31.184284307s) [upload=13959]: Got next segment. Inline: true Aug 7 23:12:56.669185727 (13h33m31.18428829s) [upload=13959]: Waiting for error group managing segments... Aug 7 23:12:56.669222629 (13h33m31.184325191s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:56.669226085 (13h33m31.18432866s) [upload=13959]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:56.669228556 (13h33m31.184331118s) [upload=13959]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.639927224 (13h34m31.155029791s) [upload=13959]: Done uploading segments... Aug 7 23:13:56.639946863 (13h34m31.155049429s) [upload=13959]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:56.754611797 (13h34m31.269714367s) [upload=13974]: Starting upload Aug 7 23:13:56.754690544 (13h34m31.269793123s) [upload=13974]: Done starting upload Aug 7 23:13:56.754787910 (13h34m31.269890478s) [upload=13974]: Uploading segments... Aug 7 23:13:56.754829279 (13h34m31.269931864s) [upload=13974]: Got next segment. Inline: true Aug 7 23:13:56.754833834 (13h34m31.269936411s) [upload=13974]: Waiting for error group managing segments... Aug 7 23:13:56.754887650 (13h34m31.269990217s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.754891993 (13h34m31.26999456s) [upload=13974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.754895495 (13h34m31.269998071s) [upload=13974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.814228870 (13h34m31.32933145s) [upload=13974]: Done uploading segments... Aug 7 23:13:56.814253732 (13h34m31.329356295s) [upload=13974]: Upload finished. err: <nil> Aug 7 23:14:51.237083244 (13h35m25.75218581s) [upload=13984]: Starting upload Aug 7 23:14:51.237154440 (13h35m25.752257016s) [upload=13984]: Done starting upload Aug 7 23:14:51.237222602 (13h35m25.752325168s) [upload=13984]: Uploading segments... Aug 7 23:14:51.237272573 (13h35m25.752375158s) [upload=13984]: Got next segment. Inline: true Aug 7 23:14:51.237277088 (13h35m25.752379654s) [upload=13984]: Waiting for error group managing segments... Aug 7 23:14:51.237328372 (13h35m25.752430937s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:51.237332974 (13h35m25.75243555s) [upload=13984]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:51.237336117 (13h35m25.752438684s) [upload=13984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:51.367347705 (13h35m25.882450281s) [upload=13984]: Done uploading segments... Aug 7 23:14:51.367368302 (13h35m25.882470865s) [upload=13984]: Upload finished. err: <nil> Aug 7 23:16:05.237199068 (13h36m39.752301655s) [upload=14002]: Starting upload Aug 7 23:16:05.237269888 (13h36m39.752372457s) [upload=14002]: Done starting upload Aug 7 23:16:05.237360382 (13h36m39.752462952s) [upload=14002]: Uploading segments... Aug 7 23:16:05.237841750 (13h36m39.752944336s) [upload=14002]: Got next segment. Inline: true Aug 7 23:16:05.237859632 (13h36m39.752962218s) [upload=14002]: Waiting for error group managing segments... Aug 7 23:16:05.237930004 (13h36m39.753032586s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:05.237936533 (13h36m39.753039109s) [upload=14002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:05.237940573 (13h36m39.753043148s) [upload=14002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:05.205359206 (13h37m39.720461782s) [upload=14002]: Done uploading segments... Aug 7 23:17:05.205376470 (13h37m39.720479046s) [upload=14002]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:15.921193047 (13h37m50.436295623s) [upload=14020]: Starting upload Aug 7 23:17:15.921258731 (13h37m50.436361294s) [upload=14020]: Done starting upload Aug 7 23:17:15.921299906 (13h37m50.436402469s) [upload=14020]: Uploading segments... Aug 7 23:17:15.921331213 (13h37m50.436433799s) [upload=14020]: Got next segment. Inline: true Aug 7 23:17:15.921334685 (13h37m50.436437248s) [upload=14020]: Waiting for error group managing segments... Aug 7 23:17:15.921366597 (13h37m50.436469172s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:15.921370316 (13h37m50.436472878s) [upload=14020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:15.921372605 (13h37m50.436475167s) [upload=14020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:15.996901858 (13h37m50.512004437s) [upload=14020]: Done uploading segments... Aug 7 23:17:15.996920869 (13h37m50.512023431s) [upload=14020]: Upload finished. err: <nil> Aug 7 23:18:10.846312036 (13h38m45.361414612s) [upload=14034]: Starting upload Aug 7 23:18:10.846377244 (13h38m45.36147981s) [upload=14034]: Done starting upload Aug 7 23:18:10.846461339 (13h38m45.361563904s) [upload=14034]: Uploading segments... Aug 7 23:18:10.846496473 (13h38m45.361599058s) [upload=14034]: Got next segment. Inline: true Aug 7 23:18:10.846501164 (13h38m45.36160374s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:18:10.846554786 (13h38m45.361657362s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:10.846559294 (13h38m45.361661871s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:10.846562922 (13h38m45.361665498s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:11.200018957 (13h38m45.715121534s) [upload=14034]: Done uploading segments... Aug 7 23:18:11.200049582 (13h38m45.715152158s) [upload=14034]: Upload finished. err: <nil> Aug 7 23:19:15.896167300 (13h39m50.411269878s) [upload=14051]: Starting upload Aug 7 23:19:15.896353995 (13h39m50.411456637s) [upload=14051]: Done starting upload Aug 7 23:19:15.896436268 (13h39m50.411538832s) [upload=14051]: Uploading segments... Aug 7 23:19:15.896501153 (13h39m50.411603732s) [upload=14051]: Got next segment. Inline: true Aug 7 23:19:15.896505978 (13h39m50.411608555s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:19:15.896570054 (13h39m50.411672631s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:15.896574888 (13h39m50.411677465s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:15.896578764 (13h39m50.411681341s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:15.976040109 (13h39m50.491142676s) [upload=14051]: Done uploading segments... Aug 7 23:19:15.976057062 (13h39m50.491159637s) [upload=14051]: Upload finished. err: <nil> Aug 7 23:19:50.253965481 (13h40m24.76906806s) [upload=14062]: Starting upload Aug 7 23:19:50.254050632 (13h40m24.769153219s) [upload=14062]: Done starting upload Aug 7 23:19:50.254136562 (13h40m24.769239133s) [upload=14062]: Uploading segments... Aug 7 23:19:50.254189310 (13h40m24.769291896s) [upload=14062]: Got next segment. Inline: true Aug 7 23:19:50.254194507 (13h40m24.769297084s) [upload=14062]: Waiting for error group managing segments... Aug 7 23:19:50.254255974 (13h40m24.769358555s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:50.254262645 (13h40m24.769365222s) [upload=14062]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:50.254267031 (13h40m24.769369607s) [upload=14062]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:50.224097554 (13h41m24.739200131s) [upload=14062]: Done uploading segments... Aug 7 23:20:50.224116508 (13h41m24.739219086s) [upload=14062]: Upload finished. err: metaclient: context deadline exceeded Duration: 268.125321ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [..........................X..XXXXX.X.XX.....................] duration: 463.341329ms FAIL (2025-08-07 23:38:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 463.371036ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [..........................XX.X.X....XX......................] duration: 272.580592ms FAIL (2025-08-07 23:38:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 272.602692ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................XXX..X.X.X.....................] duration: 567.106049ms FAIL (2025-08-07 23:38:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 567.156917ms edge gateway us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)