Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 493.719823ms FAIL (2025-06-18 18:34:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 493.741395ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 246.038042ms FAIL (2025-06-18 18:33:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 246.135802ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 513.169103ms FAIL (2025-06-18 18:33:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 513.197188ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 300.162252ms FAIL (2025-06-18 18:33:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 300.200209ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 447.822133ms FAIL (2025-06-18 18:32:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 447.850109ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 279.139471ms FAIL (2025-06-18 18:32:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 279.164768ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 474.666013ms FAIL (2025-06-18 18:31:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 474.691385ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 237.151775ms FAIL (2025-06-18 18:31:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 237.17855ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 509.741329ms FAIL (2025-06-18 18:30:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 509.781468ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 252.536659ms FAIL (2025-06-18 18:30:05 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 252.566256ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 215.253901ms FAIL (2025-06-18 18:29:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 215.281973ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 467.217448ms FAIL (2025-06-18 18:29:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 467.277289ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 254.579126ms FAIL (2025-06-18 18:28:24 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 254.619429ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 456.348422ms FAIL (2025-06-18 18:28:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 456.377158ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 273.321037ms FAIL (2025-06-18 18:27:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 273.351543ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 243.486466ms FAIL (2025-06-18 18:27:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 243.53314ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 498.838672ms FAIL (2025-06-18 18:27:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 498.935731ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 242.943203ms FAIL (2025-06-18 18:26:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 242.965755ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 453.011165ms FAIL (2025-06-18 18:26:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 453.047556ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 284.18466ms FAIL (2025-06-18 18:26:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 284.210291ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 464.387092ms FAIL (2025-06-18 18:26:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 464.410651ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 471.52306ms FAIL (2025-06-18 18:25:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 471.562664ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 251.804841ms FAIL (2025-06-18 18:25:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 251.832042ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 327.028515ms FAIL (2025-06-18 18:24:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 327.051914ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 534.786172ms FAIL (2025-06-18 18:24:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 534.881639ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 246.299309ms FAIL (2025-06-18 18:23:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 246.323461ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 471.409331ms FAIL (2025-06-18 18:23:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 471.436191ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 451.265096ms FAIL (2025-06-18 18:23:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 451.291026ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 243.426063ms FAIL (2025-06-18 18:23:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 243.476076ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 235.756229ms FAIL (2025-06-18 18:22:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 235.781516ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 278.623473ms FAIL (2025-06-18 18:22:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 278.648522ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 437.43222ms FAIL (2025-06-18 18:21:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 437.458198ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 390.742943ms FAIL (2025-06-18 18:21:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 390.767591ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 445.058157ms FAIL (2025-06-18 18:21:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 445.092459ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 219.679666ms FAIL (2025-06-18 18:21:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 219.706496ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 229.292623ms FAIL (2025-06-18 18:20:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 229.327974ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 439.33841ms FAIL (2025-06-18 18:19:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 439.370553ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 276.029031ms FAIL (2025-06-18 18:19:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 276.059562ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 451.933598ms FAIL (2025-06-18 18:19:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 451.960675ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 455.540237ms FAIL (2025-06-18 18:18:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 455.570088ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 280.469079ms FAIL (2025-06-18 18:18:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 280.515475ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 534.456676ms FAIL (2025-06-18 18:18:12 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 534.480435ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 297.111186ms FAIL (2025-06-18 18:17:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 297.13835ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 425.85609ms FAIL (2025-06-18 18:17:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 425.888781ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 236.942169ms FAIL (2025-06-18 18:16:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 236.987631ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 404.80759ms FAIL (2025-06-18 18:16:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 404.835745ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 446.20727ms FAIL (2025-06-18 18:15:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 446.233774ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 298.009ms FAIL (2025-06-18 18:15:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== questItem_SegmentMakeInline Jun 18 17:50:06.088827433 (1h26m41.652150438s) [upload=1159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:06.178822818 (1h26m41.742145826s) [upload=1159]: Done uploading segments... Jun 18 17:50:06.178860563 (1h26m41.742183565s) [upload=1159]: Upload finished. err: <nil> Jun 18 17:50:49.506686697 (1h27m25.070009697s) [upload=1166]: Starting upload Jun 18 17:50:49.506764941 (1h27m25.070087948s) [upload=1166]: Done starting upload Jun 18 17:50:49.506899239 (1h27m25.070222241s) [upload=1166]: Uploading segments... Jun 18 17:50:49.506965400 (1h27m25.070288411s) [upload=1166]: Got next segment. Inline: true Jun 18 17:50:49.506972639 (1h27m25.070295642s) [upload=1166]: Waiting for error group managing segments... Jun 18 17:50:49.507056963 (1h27m25.070379962s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:50:49.507061768 (1h27m25.070384767s) [upload=1166]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:50:49.507065097 (1h27m25.070388089s) [upload=1166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:50:49.577550167 (1h27m25.140873177s) [upload=1166]: Done uploading segments... Jun 18 17:50:49.577571110 (1h27m25.140894114s) [upload=1166]: Upload finished. err: <nil> Jun 18 17:51:35.595225340 (1h28m11.158548341s) [upload=1179]: Starting upload Jun 18 17:51:35.595321506 (1h28m11.158644506s) [upload=1179]: Done starting upload Jun 18 17:51:35.595405719 (1h28m11.158728707s) [upload=1179]: Uploading segments... Jun 18 17:51:35.595494653 (1h28m11.158817661s) [upload=1179]: Got next segment. Inline: true Jun 18 17:51:35.595499536 (1h28m11.158822536s) [upload=1179]: Waiting for error group managing segments... Jun 18 17:51:35.595561923 (1h28m11.158884922s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:51:35.595566674 (1h28m11.158889675s) [upload=1179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:51:35.595572187 (1h28m11.158895186s) [upload=1179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:51:35.686577061 (1h28m11.249900055s) [upload=1179]: Done uploading segments... Jun 18 17:51:35.686598865 (1h28m11.249921852s) [upload=1179]: Upload finished. err: <nil> Jun 18 17:52:30.037950699 (1h29m5.601273686s) [upload=1190]: Starting upload Jun 18 17:52:30.038026746 (1h29m5.601349758s) [upload=1190]: Done starting upload Jun 18 17:52:30.038124080 (1h29m5.60144707s) [upload=1190]: Uploading segments... Jun 18 17:52:30.038175858 (1h29m5.601498869s) [upload=1190]: Got next segment. Inline: true Jun 18 17:52:30.038182418 (1h29m5.601505426s) [upload=1190]: Waiting for error group managing segments... Jun 18 17:52:30.038240383 (1h29m5.60156339s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:30.038246077 (1h29m5.601569084s) [upload=1190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:30.038249894 (1h29m5.601572903s) [upload=1190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:30.118455759 (1h29m5.681778763s) [upload=1190]: Done uploading segments... Jun 18 17:52:30.118509789 (1h29m5.681832794s) [upload=1190]: Upload finished. err: <nil> Jun 18 17:52:58.463783852 (1h29m34.027106859s) [upload=1196]: Starting upload Jun 18 17:52:58.463916685 (1h29m34.027239686s) [upload=1196]: Done starting upload Jun 18 17:52:58.464026461 (1h29m34.027349456s) [upload=1196]: Uploading segments... Jun 18 17:52:58.464106903 (1h29m34.027429905s) [upload=1196]: Got next segment. Inline: true Jun 18 17:52:58.464171836 (1h29m34.027494828s) [upload=1196]: Waiting for error group managing segments... Jun 18 17:52:58.464273945 (1h29m34.027596948s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:52:58.464282267 (1h29m34.027605269s) [upload=1196]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:52:58.464287397 (1h29m34.027610397s) [upload=1196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:52:58.535770733 (1h29m34.09909374s) [upload=1196]: Done uploading segments... Jun 18 17:52:58.535794270 (1h29m34.099117271s) [upload=1196]: Upload finished. err: <nil> Jun 18 17:53:56.025315799 (1h30m31.588638789s) [upload=1209]: Starting upload Jun 18 17:53:56.025371452 (1h30m31.588694441s) [upload=1209]: Done starting upload Jun 18 17:53:56.025460666 (1h30m31.588783667s) [upload=1209]: Uploading segments... Jun 18 17:53:56.025508347 (1h30m31.588831355s) [upload=1209]: Got next segment. Inline: true Jun 18 17:53:56.025512690 (1h30m31.588835689s) [upload=1209]: Waiting for error group managing segments... Jun 18 17:53:56.025566143 (1h30m31.588889145s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:53:56.025570472 (1h30m31.588893474s) [upload=1209]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:53:56.025573644 (1h30m31.588896643s) [upload=1209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:53:56.103267225 (1h30m31.666590227s) [upload=1209]: Done uploading segments... Jun 18 17:53:56.103292645 (1h30m31.666615631s) [upload=1209]: Upload finished. err: <nil> Jun 18 17:54:22.025987742 (1h30m57.589310752s) [upload=1217]: Starting upload Jun 18 17:54:22.026141072 (1h30m57.589464081s) [upload=1217]: Done starting upload Jun 18 17:54:22.026224491 (1h30m57.58954748s) [upload=1217]: Uploading segments... Jun 18 17:54:22.026367966 (1h30m57.589690975s) [upload=1217]: Got next segment. Inline: true Jun 18 17:54:22.026376868 (1h30m57.589699878s) [upload=1217]: Waiting for error group managing segments... Jun 18 17:54:22.026432201 (1h30m57.58975521s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:54:22.026437503 (1h30m57.58976051s) [upload=1217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:54:22.026441729 (1h30m57.589764732s) [upload=1217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:54:22.096853650 (1h30m57.660176649s) [upload=1217]: Done uploading segments... Jun 18 17:54:22.096870571 (1h30m57.660193565s) [upload=1217]: Upload finished. err: <nil> Jun 18 17:55:20.540215879 (1h31m56.103538885s) [upload=1228]: Starting upload Jun 18 17:55:20.540281126 (1h31m56.103604128s) [upload=1228]: Done starting upload Jun 18 17:55:20.540349827 (1h31m56.103672828s) [upload=1228]: Uploading segments... Jun 18 17:55:20.540391558 (1h31m56.103714566s) [upload=1228]: Got next segment. Inline: true Jun 18 17:55:20.540396933 (1h31m56.103719938s) [upload=1228]: Waiting for error group managing segments... Jun 18 17:55:20.540448870 (1h31m56.103771875s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:55:20.540454027 (1h31m56.103777027s) [upload=1228]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:55:20.540457306 (1h31m56.103780305s) [upload=1228]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:55:20.609894771 (1h31m56.173217771s) [upload=1228]: Done uploading segments... Jun 18 17:55:20.609909798 (1h31m56.173232786s) [upload=1228]: Upload finished. err: <nil> Jun 18 17:56:14.635234900 (1h32m50.198557911s) [upload=1239]: Starting upload Jun 18 17:56:14.635339940 (1h32m50.198663007s) [upload=1239]: Done starting upload Jun 18 17:56:14.635464301 (1h32m50.198787303s) [upload=1239]: Uploading segments... Jun 18 17:56:14.635514535 (1h32m50.198837544s) [upload=1239]: Got next segment. Inline: true Jun 18 17:56:14.635520621 (1h32m50.19884363s) [upload=1239]: Waiting for error group managing segments... Jun 18 17:56:14.635590199 (1h32m50.198913209s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:14.635596383 (1h32m50.198919394s) [upload=1239]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:14.635600986 (1h32m50.198923995s) [upload=1239]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:14.726125378 (1h32m50.289448384s) [upload=1239]: Done uploading segments... Jun 18 17:56:14.726146507 (1h32m50.289469496s) [upload=1239]: Upload finished. err: <nil> Jun 18 17:56:36.481912768 (1h33m12.045235771s) [upload=1245]: Starting upload Jun 18 17:56:36.481999878 (1h33m12.045322886s) [upload=1245]: Done starting upload Jun 18 17:56:36.482084804 (1h33m12.045407804s) [upload=1245]: Uploading segments... Jun 18 17:56:36.482126262 (1h33m12.045449275s) [upload=1245]: Got next segment. Inline: true Jun 18 17:56:36.482132785 (1h33m12.045455782s) [upload=1245]: Waiting for error group managing segments... Jun 18 17:56:36.482179030 (1h33m12.04550203s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:56:36.482183378 (1h33m12.045506377s) [upload=1245]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:56:36.482186557 (1h33m12.045509545s) [upload=1245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:56:36.575693506 (1h33m12.13901651s) [upload=1245]: Done uploading segments... Jun 18 17:56:36.575715835 (1h33m12.139038825s) [upload=1245]: Upload finished. err: <nil> Jun 18 17:57:47.314199832 (1h34m22.877522834s) [upload=1257]: Starting upload Jun 18 17:57:47.314278019 (1h34m22.87760102s) [upload=1257]: Done starting upload Jun 18 17:57:47.314425243 (1h34m22.877748231s) [upload=1257]: Uploading segments... Jun 18 17:57:47.314465741 (1h34m22.877788751s) [upload=1257]: Got next segment. Inline: true Jun 18 17:57:47.314482135 (1h34m22.877805134s) [upload=1257]: Waiting for error group managing segments... Jun 18 17:57:47.314525737 (1h34m22.877848736s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:57:47.314530993 (1h34m22.877853991s) [upload=1257]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 18 17:57:47.314534383 (1h34m22.877857372s) [upload=1257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:47.397905504 (1h34m22.96122851s) [upload=1257]: Done uploading segments... Jun 18 17:57:47.397923220 (1h34m22.961246212s) [upload=1257]: Upload finished. err: <nil> Duration:* 298.039647ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 552.320327ms FAIL (2025-06-18 18:15:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 552.350494ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 463.061341ms FAIL (2025-06-18 18:14:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== led; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018533317 (1h33m44.581856317s) [noise=true , address=77.90.31.12:4589 , piece_id=F3LK3M7LYCOEX7VHNU3CDCXHC4GVQF5SZIZ6Y3WLGJFAOTPLUCTA, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018680893 (1h33m44.582003896s) [noise=true , address=104.36.82.7:5911 , piece_id=FNG6UCAM4GN43FVQNVRBHCXG77V2HNKXH24HBQYJWECFXFGOJTJQ, seg_pos={0 0}, upload=1253]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Jun 18 17:57:09.018711262 (1h33m44.582034266s) [seg_pos={0 0}, upload=1253]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 18 17:57:09.018818362 (1h33m44.582141372s) [upload=1253]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:57:09.018825568 (1h33m44.582148578s) [upload=1253]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:57:09.079747590 (1h33m44.6430706s) [upload=1253]: Done uploading segments... Jun 18 17:57:09.079770143 (1h33m44.643093131s) [upload=1253]: Upload finished. err: <nil> Jun 18 17:58:05.072350320 (1h34m40.63567332s) [upload=1262]: Starting upload Jun 18 17:58:05.072422163 (1h34m40.635745163s) [upload=1262]: Done starting upload Jun 18 17:58:05.072550401 (1h34m40.6358734s) [upload=1262]: Uploading segments... Jun 18 17:58:05.072879883 (1h34m40.636202873s) [upload=1262]: Got next segment. Inline: false Jun 18 17:58:05.072951305 (1h34m40.636274295s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 18 17:58:05.072956588 (1h34m40.636279581s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 18 17:58:05.105066745 (1h34m40.668389747s) [seg_pos={0 0}, upload=1262]: Begin upload segment... Jun 18 17:58:05.105115936 (1h34m40.668438936s) [seg_pos={0 0}, upload=1262]: Done begin upload segment. Jun 18 17:58:05.105156665 (1h34m40.66847967s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105169905 (1h34m40.668492906s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105558802 (1h34m40.668881806s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105640023 (1h34m40.668963028s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.105861477 (1h34m40.669184476s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106099425 (1h34m40.669422433s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106149184 (1h34m40.669472183s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106533845 (1h34m40.669856851s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106547931 (1h34m40.669870932s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.106878889 (1h34m40.670201901s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.107278936 (1h34m40.670601944s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108165905 (1h34m40.671488913s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.108612010 (1h34m40.671935028s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.109566084 (1h34m40.672889086s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.111534225 (1h34m40.674857234s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.113079164 (1h34m40.676402174s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Uploading piece... Jun 18 17:58:05.213710386 (1h34m40.777033395s) [upload=1262]: Next returned nil segment Jun 18 17:58:05.213761091 (1h34m40.77708409s) [upload=1262]: Waiting for error group managing segments... Jun 18 17:58:05.219626442 (1h34m40.782949433s) [noise=true , address=45.91.116.14:5189 , piece_id=OXTCKDFESYIGMNAUUW773QSSZZTWW5U3LJ4D3YWSHKJVWUO4OFZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.220179290 (1h34m40.783502276s) [noise=true , address=45.91.116.6:4973 , piece_id=ZK4I5CQPLE5YN66PAXXUSJMASUM2UMO4BA4Z7UYBZC7RIMDESKBQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221008680 (1h34m40.784331669s) [noise=true , address=45.91.116.5:5297 , piece_id=KFBZ6NMZXENQUVXA45JJQNOEMVLTSNL6VTY4EOFYVI2AZKVDUYQQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.221938458 (1h34m40.785261473s) [noise=true , address=104.36.82.12:6150 , piece_id=CIAQ7NTRMBA5HQ6AEQUXXDY4ADMLUBEKUBA44CSJHHAO7TC4KVCQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.222567708 (1h34m40.785890694s) [noise=true , address=45.91.116.11:5118 , piece_id=SC7B2QPYA3ZJZWZPV2ROUTVAT7YWSR3UBBZHU2BC3ZDYW4ZGL77Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223234633 (1h34m40.78655762s) [noise=true , address=77.90.31.18:4807 , piece_id=EXMN7QCNA74XVROFDSF2WELQDR6WKNAN77QSS75JNXDFMEGMSAZQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.223797627 (1h34m40.787120636s) [noise=true , address=77.90.31.13:4626 , piece_id=JQFY7DFXHLQPDI5NFY7KUTJ74MC5GLX5IE2TNQ7GV22CAMRREGAQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224283812 (1h34m40.787606801s) [noise=true , address=77.90.31.16:4733 , piece_id=H7FZFG3L3GGQMSTLPU5NAFV6U5QD6D6BULDILUDYBLLRN5RNHGKQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.224825886 (1h34m40.788148873s) [noise=true , address=77.90.31.10:4517 , piece_id=W7YVGPUGCXSWQ3G3WAFBA3CBE4FILPCWPROPJFOOCSZTSOP3JFBA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225246302 (1h34m40.788569287s) [noise=true , address=104.36.82.2:5730 , piece_id=WLHROMAZ2OCCRA2BD2OC4L72UDXPNE3ALQPKHK3LOGBEIVWJW33A, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.225697290 (1h34m40.789020285s) [noise=true , address=104.36.82.16:6294 , piece_id=XJDSVG56IAKLE37SBFEBSLUF535272I7FEITX654246GU56GNZ6Q, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226111509 (1h34m40.789434499s) [noise=true , address=104.36.82.9:6007 , piece_id=2SNQQURMPXRRDJO66S3S7EZWDPPXXCZVL6SMXGF36KHV7RYSVWEQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226490336 (1h34m40.789813326s) [noise=true , address=104.36.82.16:6295 , piece_id=DFTM2B27MX26QAERJ2Z5FOSDNH6DVVI6FLVUUIDITEBAQC77HLWQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.226498178 (1h34m40.789821168s) [seg_pos={0 0}, upload=1262]: Segment reached optimal threshold of 13 pieces. Jun 18 17:58:05.226586447 (1h34m40.789909446s) [noise=true , address=104.36.82.6:5876 , piece_id=I57NDSLJ7O2JK3DTC2V4FPYP4UMNVSXC3KCTXGNYJENX5E7L7KGA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Jun 18 17:58:05.227042761 (1h34m40.790365749s) [noise=true , address=104.36.82.2:5731 , piece_id=55Z2NETN52PIAONL5J4OWZTMIICEZFUC3I3KBQJXAWC33I7BSUIQ, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227478974 (1h34m40.790801974s) [noise=true , address=45.91.116.7:5009 , piece_id=SURS3VPNUGYGHWE4TGB74U2BRLV45G4HQXPHD3RC3RIKZXMDJFRA, seg_pos={0 0}, upload=1262]: Done uploading piece. err: <nil> Jun 18 17:58:05.227525519 (1h34m40.790848529s) [seg_pos={0 0}, upload=1262]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jun 18 17:58:05.227609653 (1h34m40.79093266s) [upload=1262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 18 17:58:05.227615973 (1h34m40.790938979s) [upload=1262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 18 17:58:05.299701748 (1h34m40.863024748s) [upload=1262]: Done uploading segments... Jun 18 17:58:05.299724403 (1h34m40.863047403s) [upload=1262]: Upload finished. err: <nil> Duration: 463.114511ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)