Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
260.001889ms
FAIL (2025-06-18 18:14:38 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 260.06995ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
232.927098ms
FAIL (2025-06-18 18:14:21 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 232.955141ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
531.777996ms
FAIL (2025-06-18 18:13:45 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 531.801019ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
283.326358ms
FAIL (2025-06-18 18:13:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 283.377532ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
437.102616ms
FAIL (2025-06-18 18:13:07 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 437.140406ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
444.989999ms
FAIL (2025-06-18 18:12:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 445.009398ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
252.420018ms
FAIL (2025-06-18 18:12:36 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 252.444123ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
436.547239ms
FAIL (2025-06-18 18:11:53 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 436.575697ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
241.000612ms
FAIL (2025-06-18 18:11:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 241.033441ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
480.903177ms
FAIL (2025-06-18 18:10:54 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 480.933886ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
251.006939ms
FAIL (2025-06-18 18:10:34 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 251.031704ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
500.864331ms
FAIL (2025-06-18 18:10:08 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 500.891891ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
280.270421ms
FAIL (2025-06-18 18:09:34 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 280.297122ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
418.504088ms
FAIL (2025-06-18 18:09:32 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 418.544414ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
286.726602ms
FAIL (2025-06-18 18:09:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 286.751037ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
491.170257ms
FAIL (2025-06-18 18:09:08 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 491.20029ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
443.893453ms
FAIL (2025-06-18 18:08:41 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 443.927237ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
246.381009ms
FAIL (2025-06-18 18:08:34 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 246.409415ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
475.855422ms
FAIL (2025-06-18 18:08:18 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 475.884709ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
230.958063ms
FAIL (2025-06-18 18:08:04 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 230.983262ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
473.538004ms
FAIL (2025-06-18 18:07:41 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 473.572836ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
248.273038ms
FAIL (2025-06-18 18:07:01 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 248.303983ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
489.875025ms
FAIL (2025-06-18 18:06:30 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 489.905265ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
219.731259ms
FAIL (2025-06-18 18:06:20 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 219.780802ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
226.467797ms
FAIL (2025-06-18 18:05:46 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 226.501992ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
499.935132ms
FAIL (2025-06-18 18:05:39 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 499.971489ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
250.767829ms
FAIL (2025-06-18 18:05:00 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 250.814269ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
445.157816ms
FAIL (2025-06-18 18:04:45 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 445.188636ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
250.308951ms
FAIL (2025-06-18 18:04:29 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 250.349413ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
251.984755ms
FAIL (2025-06-18 18:04:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 252.014554ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
497.074816ms
FAIL (2025-06-18 18:03:48 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 497.111021ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
248.910534ms
FAIL (2025-06-18 18:03:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 248.943774ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
442.11917ms
FAIL (2025-06-18 18:02:42 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 442.153177ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
334.201703ms
FAIL (2025-06-18 18:02:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 334.232333ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
481.909029ms
FAIL (2025-06-18 18:01:58 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 481.995469ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
556.873296ms
FAIL (2025-06-18 18:01:09 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 556.906928ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
235.355943ms
FAIL (2025-06-18 18:01:07 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 235.386372ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
479.25914ms
FAIL (2025-06-18 18:00:25 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 479.292548ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
235.940371ms
FAIL (2025-06-18 18:00:23 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 235.976955ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
446.415135ms
FAIL (2025-06-18 17:59:41 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 446.45774ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
236.611304ms
FAIL (2025-06-18 17:59:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 236.63906ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000557055s
FAIL (2025-06-18 17:59:32 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF
Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments...
Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil>
Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload
Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload
Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments...
Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false
Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment...
Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment.
Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece...
Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment
Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments...
Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces.
Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF
Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil>
Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments...
Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil>
Duration: 30.000581894s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000942519s
FAIL (2025-06-18 17:59:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil>
Duration: 30.000966401s
core
uplink
us-select-1
|