Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 204.518354ms FAIL (2025-08-07 17:41:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 204.606499ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 367.672466ms FAIL (2025-08-07 17:40:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 367.711011ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X...................X......X...........................] duration: 767.128559ms FAIL (2025-08-07 17:40:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 767.153922ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 528.137286ms FAIL (2025-08-07 17:39:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 528.16705ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X...................X......X..........................] duration: 584.718399ms FAIL (2025-08-07 17:39:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 584.762636ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 199.269046ms FAIL (2025-08-07 17:38:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 199.297784ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X...................X......X.........................] duration: 569.099097ms FAIL (2025-08-07 17:38:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 569.13188ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 438.549677ms FAIL (2025-08-07 17:37:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 438.568787ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X...................X......X........................] duration: 956.211531ms FAIL (2025-08-07 17:37:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 956.240992ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 214.360008ms FAIL (2025-08-07 17:37:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 214.397311ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X...................X......X.......................] duration: 668.11323ms FAIL (2025-08-07 17:36:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 668.136627ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 227.619699ms FAIL (2025-08-07 17:36:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 227.639845ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 345.294612ms FAIL (2025-08-07 17:35:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 345.384273ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X...................X......X......................] duration: 724.267751ms FAIL (2025-08-07 17:35:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 724.292421ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X...................X......X.....................] duration: 759.089703ms FAIL (2025-08-07 17:35:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 759.118874ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 385.717273ms FAIL (2025-08-07 17:35:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 385.742845ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 272.652712ms FAIL (2025-08-07 17:34:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 272.677563ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X...................X......X....................] duration: 546.224137ms FAIL (2025-08-07 17:33:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 546.253486ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X...................X......X...................] duration: 845.298725ms FAIL (2025-08-07 17:33:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 845.321209ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 225.639938ms FAIL (2025-08-07 17:32:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 225.665252ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X...................X......X..................] duration: 502.467797ms FAIL (2025-08-07 17:32:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 502.499277ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X...................X......X.................] duration: 440.572326ms FAIL (2025-08-07 17:32:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 440.597786ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 265.814702ms FAIL (2025-08-07 17:32:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 265.840752ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 485.406723ms FAIL (2025-08-07 17:31:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 485.449297ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X...................X......X................] duration: 580.121608ms FAIL (2025-08-07 17:31:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 580.152663ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 282.573367ms FAIL (2025-08-07 17:30:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 282.599641ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X...................X......X...............] duration: 722.112909ms FAIL (2025-08-07 17:30:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 722.147071ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 216.830663ms FAIL (2025-08-07 17:30:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 216.865058ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X...................X......X..............] duration: 1.011678981s FAIL (2025-08-07 17:30:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 1.011709486s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 413.997335ms FAIL (2025-08-07 17:29:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 414.024865ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X...................X......X.............] duration: 1.224132945s FAIL (2025-08-07 17:29:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 1.224156237s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 382.772776ms FAIL (2025-08-07 17:29:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 382.875728ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 195.520661ms FAIL (2025-08-07 17:28:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 195.547839ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X...................X......X............] duration: 538.867112ms FAIL (2025-08-07 17:28:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 538.905875ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X...................X......X...........] duration: 859.014167ms FAIL (2025-08-07 17:28:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 859.039175ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 574.340377ms FAIL (2025-08-07 17:27:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 574.365232ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X...................X......X..........] duration: 674.607667ms FAIL (2025-08-07 17:27:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 674.6337ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 379.572527ms FAIL (2025-08-07 17:27:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 379.659967ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 341.880362ms FAIL (2025-08-07 17:26:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 341.904492ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...................X......X.........] duration: 509.068619ms FAIL (2025-08-07 17:26:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 509.099285ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X...................X......X........] duration: 501.163133ms FAIL (2025-08-07 17:25:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 501.202748ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X...................X......X.......] duration: 499.845107ms FAIL (2025-08-07 17:24:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 499.954161ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X...................X......X......] duration: 779.945953ms FAIL (2025-08-07 17:23:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 779.973767ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X...................X......X.....] duration: 580.808425ms FAIL (2025-08-07 17:22:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 580.839096ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X...................X......X....] duration: 481.435509ms FAIL (2025-08-07 17:21:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 481.461999ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X...................X......X...] duration: 1.801727325s FAIL (2025-08-07 17:21:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 1.801750277s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X...................X......X..] duration: 584.056298ms FAIL (2025-08-07 17:21:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 584.08277ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X...................X......X.] duration: 518.651101ms FAIL (2025-08-07 17:20:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 518.675919ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X...................X......X] duration: 43.450403ms FAIL (2025-08-07 17:19:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 43.479062ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X...................X......] duration: 600.597249ms FAIL (2025-08-07 17:19:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== .BatchRequestItem_ObjectBegin Aug 7 17:14:05.445383605 (7h34m39.960486186s) [upload=7866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:14:05.534779439 (7h34m40.049882018s) [seg_pos={0 0}, upload=7866]: Begin upload segment... Aug 7 17:14:05.534857801 (7h34m40.049960394s) [seg_pos={0 0}, upload=7866]: Done begin upload segment. Aug 7 17:14:05.534895203 (7h34m40.049997793s) [noise=true , address=104.36.82.46:4493 , piece_id=IILK6KWT3DDEPEPTSFW743JRMUWEZD7MITKEDRXUYBC5XWA6ODNA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.534908646 (7h34m40.050011213s) [noise=true , address=104.36.82.39:4759 , piece_id=42A2R3LCI3C7Q25XWBZ532DZBFEKKHKCSO4CJS73AB6ZR5U7QMOA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.534994375 (7h34m40.050096957s) [noise=true , address=64.49.13.155:4920 , piece_id=U2AHNYZQU2OAJZ7JFR7HDDP4DKSE5MZNDQKW627PJ56T37WB3E4A, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.535467307 (7h34m40.050569895s) [noise=true , address=104.36.82.5:4870 , piece_id=S4QTI7S2ZAH7L5QRIPIMFW3I6SJV7GQNNEHMCC7B6DN27UYDOMDA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.535564340 (7h34m40.050666948s) [noise=true , address=104.36.82.53:4136 , piece_id=QRFIY2UJEIYW3264RFEBUSMHQGTW5RIQO7XGWCZUC3ZERW35AFVA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.535843313 (7h34m40.0509459s) [noise=true , address=104.36.82.55:4194 , piece_id=YM3I2HNCDATQGUFN5NSHKT6OOBQQG3YQCKXJJ2ZAWW3KDQRQLRJA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.536163752 (7h34m40.051266335s) [noise=true , address=45.91.116.8:4054 , piece_id=VNVXAXAWHVT3RWYOBBJYQA6FA346IWVO2VELQBGIMP6YLBT2IGZA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.536308887 (7h34m40.051411473s) [noise=true , address=104.36.82.16:4319 , piece_id=K2CST26EBN4C3K5L7GZWETTCRWZCYOHIGH72OIAATONVHNOTVBWQ, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.536637925 (7h34m40.051740506s) [noise=true , address=104.36.82.10:4103 , piece_id=IFABLPQQB24R3LDY3RJY3WQNYOVYOXLXEDUPOJ6U3QK4WFYTCVMA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.536655240 (7h34m40.051757827s) [noise=true , address=104.36.82.17:4332 , piece_id=3BCE6NUYSFGKMUJ42RANWJJXLOFPA7EAVYAKH2PRLGQVS2MJMKRA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.536947657 (7h34m40.052050236s) [noise=true , address=104.36.82.60:4368 , piece_id=L6WGEZ4XTAJVZ6U4ADMRDHJN4TZHXN4W6TJWAXIP6HOPWYH5RJAA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.537118188 (7h34m40.052220773s) [noise=true , address=104.36.82.58:4274 , piece_id=JTNNIQ2ZTS644NFSYIZFXWGOWBJD3QREVI4NZMOJIM7HW7GWATIA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.537269378 (7h34m40.052371954s) [noise=true , address=45.91.116.13:4184 , piece_id=EQKTKDL4DIYMBVIIOZWKBBYPKLJODUZTCK6R4WVNJI3B5GG5XDQA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.537662702 (7h34m40.052765291s) [noise=true , address=104.36.82.43:4848 , piece_id=5WAO2INQLCOUMQZCF5PDN65ZWGMNAXCVXBO3BOY7IHOBZWJ3L47Q, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.539440271 (7h34m40.05454285s) [noise=true , address=79.127.205.233:4632 , piece_id=QGJ4X6S47F7QHUJHCEFONI5QOVK4TEXMRNPRMVGWVDGFD5O4GXAQ, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.540326255 (7h34m40.055428841s) [noise=true , address=104.36.82.59:4308 , piece_id=GSTWR56AKN5YQ2ECWYQUUC5TE6NDU2XST4SUWZ65TUJW3VN7ZNXA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.540862680 (7h34m40.055965262s) [noise=true , address=104.36.82.9:4017 , piece_id=MCBUBFETS3QI64UWWXBBF3OOZEUUNJF5USMEERLV5DMJBQMXI6OA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.542037241 (7h34m40.057139819s) [noise=true , address=64.49.13.26:4952 , piece_id=SO7LPGTIPPKUV6ZTQIRX6H2UP3I365P6HMAO5HGFOS33HOEPAXEA, seg_pos={0 0}, upload=7866]: Uploading piece... Aug 7 17:14:05.648934634 (7h34m40.164037219s) [upload=7866]: Next returned nil segment Aug 7 17:14:05.648953871 (7h34m40.164056457s) [upload=7866]: Waiting for error group managing segments... Aug 7 17:14:05.656881070 (7h34m40.171983636s) [noise=true , address=104.36.82.55:4194 , piece_id=YM3I2HNCDATQGUFN5NSHKT6OOBQQG3YQCKXJJ2ZAWW3KDQRQLRJA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.661055993 (7h34m40.176158559s) [noise=true , address=104.36.82.39:4759 , piece_id=42A2R3LCI3C7Q25XWBZ532DZBFEKKHKCSO4CJS73AB6ZR5U7QMOA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.665139330 (7h34m40.180241896s) [noise=true , address=45.91.116.13:4184 , piece_id=EQKTKDL4DIYMBVIIOZWKBBYPKLJODUZTCK6R4WVNJI3B5GG5XDQA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.673362182 (7h34m40.18846475s) [noise=true , address=45.91.116.8:4054 , piece_id=VNVXAXAWHVT3RWYOBBJYQA6FA346IWVO2VELQBGIMP6YLBT2IGZA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.674065570 (7h34m40.189168135s) [noise=true , address=104.36.82.53:4136 , piece_id=QRFIY2UJEIYW3264RFEBUSMHQGTW5RIQO7XGWCZUC3ZERW35AFVA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.674464549 (7h34m40.189567127s) [noise=true , address=104.36.82.46:4493 , piece_id=IILK6KWT3DDEPEPTSFW743JRMUWEZD7MITKEDRXUYBC5XWA6ODNA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.674534531 (7h34m40.189637101s) [noise=true , address=104.36.82.9:4017 , piece_id=MCBUBFETS3QI64UWWXBBF3OOZEUUNJF5USMEERLV5DMJBQMXI6OA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.675321914 (7h34m40.190424485s) [noise=true , address=104.36.82.43:4848 , piece_id=5WAO2INQLCOUMQZCF5PDN65ZWGMNAXCVXBO3BOY7IHOBZWJ3L47Q, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.678820294 (7h34m40.19392286s) [noise=true , address=104.36.82.60:4368 , piece_id=L6WGEZ4XTAJVZ6U4ADMRDHJN4TZHXN4W6TJWAXIP6HOPWYH5RJAA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.690917968 (7h34m40.206020546s) [noise=true , address=104.36.82.5:4870 , piece_id=S4QTI7S2ZAH7L5QRIPIMFW3I6SJV7GQNNEHMCC7B6DN27UYDOMDA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.693097507 (7h34m40.2082001s) [noise=true , address=104.36.82.17:4332 , piece_id=3BCE6NUYSFGKMUJ42RANWJJXLOFPA7EAVYAKH2PRLGQVS2MJMKRA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.697148924 (7h34m40.212251486s) [noise=true , address=104.36.82.10:4103 , piece_id=IFABLPQQB24R3LDY3RJY3WQNYOVYOXLXEDUPOJ6U3QK4WFYTCVMA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.697919267 (7h34m40.213021836s) [noise=true , address=104.36.82.58:4274 , piece_id=JTNNIQ2ZTS644NFSYIZFXWGOWBJD3QREVI4NZMOJIM7HW7GWATIA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.818479195 (7h34m40.333581772s) [noise=true , address=104.36.82.16:4319 , piece_id=K2CST26EBN4C3K5L7GZWETTCRWZCYOHIGH72OIAATONVHNOTVBWQ, seg_pos={0 0}, upload=7866]: Done uploading piece. err: <nil> Aug 7 17:14:05.818496748 (7h34m40.333599324s) [seg_pos={0 0}, upload=7866]: Segment reached optimal threshold of 14 pieces. Aug 7 17:14:05.818850686 (7h34m40.333953263s) [noise=true , address=104.36.82.59:4308 , piece_id=GSTWR56AKN5YQ2ECWYQUUC5TE6NDU2XST4SUWZ65TUJW3VN7ZNXA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tf9N11h9nLjVCMDpqacHE4qT6NYoPAGWLPbkLX14TZPp8dUV9): protocol: write tcp 159.223.143.39:36616->104.36.82.59:4308: use of closed network connection; write tcp 159.223.143.39:36616->104.36.82.59:4308: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36616->104.36.82.59:4308: use of closed network connection Aug 7 17:14:05.819198457 (7h34m40.334301035s) [noise=true , address=79.127.205.233:4632 , piece_id=QGJ4X6S47F7QHUJHCEFONI5QOVK4TEXMRNPRMVGWVDGFD5O4GXAQ, seg_pos={0 0}, upload=7866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V6JKTP7zarv1WrXD7giosej1eFWCEhUkDok4xS7obfZy4ksbw): protocol: read tcp 159.223.143.39:40672->79.127.205.233:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 17:14:05.819265015 (7h34m40.334367599s) [noise=true , address=64.49.13.155:4920 , piece_id=U2AHNYZQU2OAJZ7JFR7HDDP4DKSE5MZNDQKW627PJ56T37WB3E4A, seg_pos={0 0}, upload=7866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1frFBZNqMLDUjojkdhrBU4GFXckd3kB2Seok1yXngUT7UyciBz): protocol: read tcp 159.223.143.39:53422->64.49.13.155:4920: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 17:14:05.819306255 (7h34m40.334408832s) [noise=true , address=64.49.13.26:4952 , piece_id=SO7LPGTIPPKUV6ZTQIRX6H2UP3I365P6HMAO5HGFOS33HOEPAXEA, seg_pos={0 0}, upload=7866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1okyMVGQotKJf1ysn8KeBxsmQJoR1amH1kLtYpgYGnADRcK5C8): protocol: read tcp 159.223.143.39:35222->64.49.13.26:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 17:14:05.819330501 (7h34m40.334433065s) [seg_pos={0 0}, upload=7866]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:14:05.819405469 (7h34m40.334508047s) [upload=7866]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:14:05.819411338 (7h34m40.334513918s) [upload=7866]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:14:05.930048860 (7h34m40.445151445s) [upload=7866]: Done uploading segments... Aug 7 17:14:05.930069948 (7h34m40.445172531s) [upload=7866]: Upload finished. err: <nil> Duration:* 600.619955ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)