Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 518.536897ms FAIL (2025-07-19 16:45:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 518.568677ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 447.890922ms FAIL (2025-07-19 16:44:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 447.929982ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 531.443792ms FAIL (2025-07-19 16:44:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 531.49753ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 489.196967ms FAIL (2025-07-19 16:43:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 489.299571ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 721.846831ms FAIL (2025-07-19 16:42:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 721.882068ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 677.460981ms FAIL (2025-07-19 16:41:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 677.510849ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 462.89768ms FAIL (2025-07-19 16:41:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 462.931439ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 480.599676ms FAIL (2025-07-19 16:40:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 480.652826ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 495.431721ms FAIL (2025-07-19 16:39:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 495.529274ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 430.770045ms FAIL (2025-07-19 16:38:03 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 430.802797ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 489.742868ms FAIL (2025-07-19 16:37:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 489.78083ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 561.139858ms FAIL (2025-07-19 16:36:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 561.167921ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 620.71582ms FAIL (2025-07-19 16:34:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 620.747732ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 572.956992ms FAIL (2025-07-19 16:33:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 572.993606ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 504.780922ms FAIL (2025-07-19 16:33:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 504.816391ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 657.540516ms FAIL (2025-07-19 16:32:03 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 657.626968ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 470.943572ms FAIL (2025-07-19 16:31:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 470.979274ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 454.00004ms FAIL (2025-07-19 16:30:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 454.028045ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 484.317499ms FAIL (2025-07-19 16:29:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 484.346828ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 521.637945ms FAIL (2025-07-19 16:28:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 521.668381ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 450.786323ms FAIL (2025-07-19 16:27:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 450.845557ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 501.874982ms FAIL (2025-07-19 16:26:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 501.908419ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 486.997957ms FAIL (2025-07-19 16:25:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 487.030951ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 477.247463ms FAIL (2025-07-19 16:24:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 477.281644ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 468.114085ms FAIL (2025-07-19 16:23:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 468.22793ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 371.329223ms FAIL (2025-07-19 16:23:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 371.362614ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 518.819116ms FAIL (2025-07-19 16:22:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 518.858227ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 589.584115ms FAIL (2025-07-19 16:22:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 589.685936ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 455.739857ms FAIL (2025-07-19 16:21:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 455.770632ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 487.979989ms FAIL (2025-07-19 16:21:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 488.006617ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 577.138195ms FAIL (2025-07-19 16:20:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 577.175879ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 646.470286ms FAIL (2025-07-19 16:19:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 646.523251ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 478.776674ms FAIL (2025-07-19 16:18:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 478.813664ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 515.657441ms FAIL (2025-07-19 16:18:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 515.693638ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 473.075949ms FAIL (2025-07-19 16:18:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 473.147137ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 468.601183ms FAIL (2025-07-19 16:17:24 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 468.629909ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 517.675451ms FAIL (2025-07-19 16:16:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 517.722427ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 635.864695ms FAIL (2025-07-19 16:15:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 635.904889ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 989.484445ms FAIL (2025-07-19 16:15:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 989.518044ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 440.170798ms FAIL (2025-07-19 16:14:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 440.203598ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 439.474458ms FAIL (2025-07-19 16:13:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 439.565018ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 554.492133ms FAIL (2025-07-19 16:13:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 554.52906ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001748078s FAIL (2025-07-19 16:12:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== JAS3NAEHPQ, seg_pos={0 0}, upload=13805]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: expected piece hash; context canceled; EOF Jul 19 16:10:50.664428623 (17h20m32.513035707s) [seg_pos={0 0}, upload=13805]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:10:50.664509198 (17h20m32.513116285s) [upload=13805]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:10:50.664514760 (17h20m32.513121827s) [upload=13805]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:10:50.736789321 (17h20m32.585396411s) [upload=13805]: Done uploading segments... Jul 19 16:10:50.736826215 (17h20m32.585433295s) [upload=13805]: Upload finished. err: <nil> Jul 19 16:11:58.314006398 (17h21m40.162613482s) [upload=13817]: Starting upload Jul 19 16:11:58.314749584 (17h21m40.16335667s) [upload=13817]: Done starting upload Jul 19 16:11:58.315040835 (17h21m40.163647924s) [upload=13817]: Uploading segments... Jul 19 16:11:58.315354232 (17h21m40.16396132s) [upload=13817]: Got next segment. Inline: false Jul 19 16:11:58.315426816 (17h21m40.164033905s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:11:58.315434002 (17h21m40.164041082s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:11:58.349934256 (17h21m40.198541338s) [seg_pos={0 0}, upload=13817]: Begin upload segment... Jul 19 16:11:58.349982513 (17h21m40.198589598s) [seg_pos={0 0}, upload=13817]: Done begin upload segment. Jul 19 16:11:58.350018513 (17h21m40.198625595s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350588007 (17h21m40.199195095s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.350804146 (17h21m40.199411237s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351514170 (17h21m40.200121264s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.351771227 (17h21m40.200378315s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352670435 (17h21m40.20127752s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.352723228 (17h21m40.201330314s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.353502192 (17h21m40.202109274s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.354132630 (17h21m40.202739715s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.356556981 (17h21m40.205164061s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.357776003 (17h21m40.206383088s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.360436506 (17h21m40.209043598s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.362556012 (17h21m40.211163105s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.365515091 (17h21m40.214122216s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.383635044 (17h21m40.232242128s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.384662269 (17h21m40.233269351s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Uploading piece... Jul 19 16:11:58.476689441 (17h21m40.325296511s) [upload=13817]: Next returned nil segment Jul 19 16:11:58.476708926 (17h21m40.325315996s) [upload=13817]: Waiting for error group managing segments... Jul 19 16:11:58.480780753 (17h21m40.329387833s) [noise=true , address=45.91.116.12:4225 , piece_id=OV5YKOGQRWBHCHSGDWE6PCOXKQJD2WLPFXYKBZV4SATEQL57J23Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.482043942 (17h21m40.330651012s) [noise=true , address=77.90.31.12:4589 , piece_id=Y4RBW4INNW5HBWR4BUUU6Z7EJCKJPIOYU2QJXFEQ7O5RJXIQSL2Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.484678573 (17h21m40.333285658s) [noise=true , address=45.91.116.7:4011 , piece_id=I3HNWXHWLEQGFJB7PTS5SHME5VVMR2OP2DTPSCCAUPAWZEPNOEDA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485257307 (17h21m40.333864389s) [noise=true , address=45.91.116.4:4938 , piece_id=7Z4MSRXBGSCVXARUZ4WBFKFSJRXAUCIQDC72J2FB4AYL4K2OZNPA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.485858320 (17h21m40.334465402s) [noise=true , address=45.91.116.5:4298 , piece_id=BGNVOK6Q67PU5H5ALPXYDXBWUXNMFFP7KQKUV7TT2KRPAAO35O6Q, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486366983 (17h21m40.334974063s) [noise=true , address=45.91.116.13:4153 , piece_id=ZMLT5WM7H7IQ3RHHU6CSXGXRLERF3RWWWWHGPA7TDAS3BETGNNQQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.486892050 (17h21m40.335499132s) [noise=true , address=45.91.116.4:4940 , piece_id=QYXIUAGE6W2VBGNZIPLOQJ5HPRTQ4I5SLX74OJUXBFL7SP63QZGQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487479760 (17h21m40.336086842s) [noise=true , address=77.90.31.10:4518 , piece_id=C5KB3C7FEJYBWU43664CEQF4ICN2UJXHGVB3SWRIQZRUE3NC6IAQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.487681707 (17h21m40.336288776s) [noise=true , address=45.91.116.14:4191 , piece_id=EB6TGUJJWKD667EIB5FA7D3GAS6XZLPW5NBZUFNGWNVKWRFLOZZA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488328759 (17h21m40.336935845s) [noise=true , address=77.90.31.15:4699 , piece_id=7DNHHOXWMCVZQZ2NJ7QYW22VYSUC44HMUS5TJUQZSVCJVT3L4ADQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488375781 (17h21m40.336982854s) [noise=true , address=45.91.116.9:4262 , piece_id=37BND3Y46FZYULQ3VWYYQL4HTYVF26QOTY2W2UBZVBGE7HJ34QNA, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.488895257 (17h21m40.33750234s) [noise=true , address=104.36.82.6:4876 , piece_id=2K4CLNEXGGP3N4DF4YRWISAUJY2RPD2ZNKGH6WOD7WBF4FD5DOIQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495361446 (17h21m40.34396853s) [noise=true , address=104.36.82.14:4222 , piece_id=MUEYBPLWRIL7O6TRUIALB7QCV6NWBR24JAHCQIJECXH7EX6CESHQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: <nil> Jul 19 16:11:58.495390833 (17h21m40.343997919s) [seg_pos={0 0}, upload=13817]: Segment reached optimal threshold of 13 pieces. Jul 19 16:11:58.495569782 (17h21m40.344176865s) [noise=true , address=104.36.82.13:4187 , piece_id=CEKQJDKH3D5KGFMLU4UGHEO23OFXPB46YFOQNLWUKEUZQ2JFMJ2A, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495654740 (17h21m40.344261935s) [noise=true , address=104.36.82.9:4005 , piece_id=7EH7CSLKKWXZEHDX7NTECU5L5EJ4L4VIEUSX4JBNF6AGCHSJJ6LQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 19 16:11:58.495872315 (17h21m40.344479403s) [noise=true , address=104.36.82.10:4078 , piece_id=AJ4HEQ5WBS7SZRLRTSXBYMPRYQD2GF545MIDXA3QM5M3IFDOG5FQ, seg_pos={0 0}, upload=13817]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:32812->104.36.82.10:4078: use of closed network connection Jul 19 16:11:58.495925334 (17h21m40.34453242s) [seg_pos={0 0}, upload=13817]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:11:58.496062793 (17h21m40.344669888s) [upload=13817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:11:58.496071020 (17h21m40.344678104s) [upload=13817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:11:58.547447138 (17h21m40.396054224s) [upload=13817]: Done uploading segments... Jul 19 16:11:58.547468806 (17h21m40.39607588s) [upload=13817]: Upload finished. err: <nil> Duration: 30.00179199s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 32.900546ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 22:50:10 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 32.928255ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 13.532086ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 22:49:33 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 13.554931ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 15.664625ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 22:49:19 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 15.697187ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 14.2364ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 22:49:09 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 14.260669ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 59.604234ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 22:48:55 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 59.629298ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 37.987009ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 22:48:47 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 38.018179ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 14.264227ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 22:47:55 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 14.295145ms edge ipfs us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)